4

I'm writing an InputStream that processes data streams containing "trailer" data. That is, the final n bytes of the stream is a piece of data that must be handled separately and shouldn't be returned from the read methods.

My class will maintain an internal byte array of length n and read data into this byte array from the underlying stream. Data will then be popped from the front of the buffer to service the requests to read.

Implementing the int read() method is simple enough - I just call the same method on the underlying stream, shift my byte array by 1 and pop the result to the caller.

Implementing int read(byte[] b) is more challenging as it requires shifting a large number of bytes through my n-byte buffer, whilst popping and collating the bytes from the front. Essentially a "sliding window" buffer, always containing the last n bytes from my underlying stream.

I get the feeling I'm re-inventing the wheel if I start coding this from scratch. Can anyone suggest if I've overlooked an existing InputStream or Queue implementation that would be well suited to this task?

Improve this question
  • I would be interested to know the answer to this. Maybe there is some stream used for http which does this sort of thing. – Neil Apr 29 '13 at 8:34
2

Use a ring buffer of size n. I don't suppose there is a ready-made implementation, but it isn't that complicated to do yourself. As you intend to retain n bytes at all times, after initialization you don't need to worry about the buffer being partially filled (so you probably only need one read/write pointer, rather than one read pointer and one write pointer). Reading k bytes is simple:

  • Copy the k bytes following the r/w pointer to the output buffer (special case k = 1 for read()).
  • Read k bytes from the underlying stream and adjust the r/w pointer.

When the underlying stream runs out, you'll have a n-element array with the last n bytes starting from the r/w pointer. This is extremely fast too - no need to shift whole arrays around. One downside is that processing the trailer is not as trivial as if it was an array (but still simple).

Improve this answer
  • This looks like a nice approach - I'll give it a go. – Duncan Jones Apr 29 '13 at 14:17
0

Don't shift your buffer, it is not a linked queue. Use an integer as an index, which you increment as you read data out. When the index is equal to the length of the byte array, you read more data.

Improve this answer
  • Because I never know when the underlying stream will finish, I must maintain n unreturned bytes within my class at all times. Can that constraint be met with your solution idea? Also, while your implementation comments are welcome, I'm particularly interested to see if I've overlooked existing code that acts in a similar fashion. – Duncan Jones Apr 29 '13 at 11:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.