How to write multi-format parsers?

Question

I'm about to create a specific file format for a particular project.

There is no special metadata related to the file, as it can be saved in the database, in a file, etc. So the extension of the file, the date, etc. don't matter at all: only the content does.

I now know that the format will have several iterations, so the first thing I'm doing is designing it with further versions in mind.

What I've thought is the following:

<format-id>       -> 4 bytes
<format-version>  -> 1 byte

Now the hard part: when I want to parse any version of the format, what should I do?

I've thought about using a PushbackInputStream, reading the first 5 bytes and fetch the appropriate version parser on the basis of those bytes, then unread the bytes and present the "pristine" stream to the actual parser. This way I can test parsers independently without using my "parser-aggregator".

Example:

byte[] identifier = "ABCD".getBytes(UTF_8);
try (PushbackInputStream in = ...) {
  byte[] header = IOUtils.readFully(in, new byte[5]);
  in.unread(header);
  if (!Arrays.equals(identifier, Arrays.copyOf(header, 4))) {
    throw new UnknownFormatException();
  }
  int version = header[4];
  getParser(version).parse(in);
}

The second way I've thought of is to present the start of stream to the parser and let it accept or refuse to parse the rest of the stream:

try (PushbackInputStream in = ...) {
  byte[] header = IOUtils.readFully(in, new byte[5]);
  in.unread(header);
  for (Parser parser: parsers) {
    if (parser.accept(header)) {
      parser.parse(in);
      break;
    }
  }
}

My question is: are there robust techniques to implement such a multi-format parser? For instance, how does Java Swing's ImageIO handle so many formats?

Notes:

• I'm mostly trying to avoid each parser to try to parse the stream. I absolutely don't want that to happen. This question answers that, but that's not what I want.

Answer 1

Why bother re-parsing the header? By the time your version-specific parser has been instantiated, you already know what the header must have contained, so just parse the body.

If the header contains some other information the body parser needs, just pass it as an argument when you instantiate the body parser.

You can think of this as an encapsulated format [header][body], where the header tells you which body parser to use. If you need more metadata, the body can contain a nested [header2][body2], or whatever other format you need.

Answer 2

Create a stream. Pass it to a miniature parser that only looks at the format and version. Then re-create the stream and pass it to the correct parser.

This way, you don't instantiate any heavy-weight parsers until the need arises.