Kind of a functional programming newbie question here:
I've been reading the transcripts of some of Rich Hickey's talks, and in several of his more well-known ones, he recommends using queues as an alternative to having functions call one another. (E.g. in Design, Composition and Performance and in Simple Made Easy.)
I don't quite understand this, in a number of respects:
Is he talking about putting data in a queue and then having each function use it? So instead of function A calling function B to carry out its own computation, we just have function B slap its output on a queue and then have function A grab it? Or, alternatively, are we talking about putting functions on a queue and then successively applying them to data (surely not, because that would involve massive mutation, right? And also multiplication of queues for multiple-arity functions, or like trees or something?)
How does that make things simpler? My intuition would be that this strategy would create more complexity, because the queue would be a kind of state, and then you have to worry "what if some other function snuck in and put some data on top of the queue?"
One answer to an implementation question on SO suggests that the idea is creating a bunch of different queues. So each function puts its output in its own queue(??). But that also confuses me, because if you're running a function once, then why does it need a queue for its output when you could just take that output and slap a name on it as a (var, atom, entry in a big hash table, whatever). By contrast, if a function is running multiple times, and you stick its output onto a queue, then you've inflicted state on yourself again, and you have to worry about the order in which everything is called, downstream functions get less pure, etc.
Clearly I'm not understanding the point here. Can someone explain a little?
Job
object, push that into a queue, and have one or more worker threads work on that queue. TheJob
then dispatches moreJob
s into the queue upon completion. Return values are replaced by callbacks in that concept. It's a nightmare to debug and verify as you lack a call stack, and efficient and flexible for the very same reason.