What's the difference between a stream and a queue? They both have the concept of an ordered set of elements, but tend to have different implementations and a different vocabulary of 'insert'/'extract' (streams) vs. 'enqueue'/'dequeue' (queue). Are these interchangable? Do they suggest different concepts or patterns? If so, what are the differences?
A stream is not really a data structure as such (conceptually), but is a sequence of digitally encoded coherent signals (packets of data or data packets) used to transmit or receive information". So basically a sequence of data.
A queue is a simple FIFO mechanism allowing you to add items to the back of the queue or take from the front.
Streams always have a source, e.g. a file, network location, etc. A Queue does not inherently contain any data.
So essentially they are quite different in concept and as Mason pointed out, they are used differently.
The basic difference is in the way they're used. In a stream, you usually only use one side of the operation: you open a stream to read, or write, but not both. Whereas with a queue, you're putting items on and taking them off.
Also, queues are very strict about the order in which you put things on and take them off, while streams often (but not always) support a
Seek operation, especially if you're reading from them.
In my experience, a stream is a sequence of bytes that are produced/consumed at rate often determined by data within the stream. For example, an MPEG data stream will have frame headers which describe what the next sequence of bytes does and how many need to be consumed. Binary serialisation of a document would be similar. It is not always self-describing : writing to STDOUT can be done in a stream-wise manner but it may be humanly-readable/non-parseable data.
Conversely, a queue is usually of a well-known type of object (or interface-supporting objects) that are consumed in their entirety. An example might be a queue of database jobs that are processed by a number of database workers.
A difference between a stream and a queue is the way data rate is controlled:
in a queue, the sender adapts to the speed of the reader. The sender decides what to do if the queue is full: wait for availability of queue or throw data away.
in a stream, the reader adapts to the speed of the sender, The reader decides what to do if new data arrives before old one has been consumed.
With that perspective, character streams such as Unix pipes would not qualify as streams but as queues.
If we think more visually about how the words are commonly used, we can avoid the clutter of specific uses by particular languages and implementations, so that these terms can actually mean something:
- A queue of people waits in line and are served one by one. More people join the queue at the tail. Everyone waits as service proceeds and the service time is expected to vary. You can speak of how many people are served in total.
- A stream of people, for example leaving a building through a door, are not served one by one, they just pass the exit point at a more-or-less steady rate. Delays are not expected and not well tolerated. You can speak of a rate of people: one per second.
That is the intention of these terms. They are metaphors. (like everything else) (Shhh! you'll ruin the story!)
A queue is a higher level concept than a stream. The base elements of a queue is a message/object, which is a coherent (usually typed) data structure that can be interpreted by the consumer on its own. On the other hand, at the base of a stream, there is (usually fixed-size) bits/bytes/characters, which, on its own, is usually meaningless to the application. A sequence of these characters may compose a "message", but a stream API leaves it to the application to split the sequence of characters into sensible chunks.
A Stream API usually also allows partial reads and writes, if the stream buffers are full and the other side isn't reading/writing; applications that handles queues usually expects the queue to handle these internally.
A queue may be implemented on top of a stream, this is done by implementing message framing. For example, TCP provides a stream interface, HTTP is built on top of TCP, and adds message framing using Content-Length/chunked transfer encoding. Users of an HTTP Connection API are abstracted from dealing with splitting HTTP connection stream to HTTP requests.
On the other hand, usually, it makes less sense to implement a stream API on top of a queue, as the handling of message framing adds unnecessary overhead.
In functional programming languages (e.g. Scala), and maybe other languages as well, streams are really more like functional lists and they are queues. I should note, however, queues can actually be implemented using a pair of lists. In Scala and probably elsewhere, a Stream is just a lazy List - more specifically, the tail of the list is a
Functional streams can share some similarity with queues, as opposed to, lists, in that you can use them in a way that you don't keep a reference to the head of the stream - but you have to be careful: https://stackoverflow.com/a/5159356/3096687. This is somewhat analogous to a dequeue call to a queue (although in the case of a stream, you do so implicitly: http://daily-scala.blogspot.com/2010/01/streams-2-stream-construction.html).