Would you use Byte Arrays or XML if size was not an issue?

Question

In my basic understanding of ByteArrays the benefit is that it is smaller in file size.

The down size of a ByteArray is that for any given format you have to know the about the file format to get information out of it. You need a specification and software tools or software to find information.

For example, to get information about a JPEG you have to know what to look for (markers) and have knowledge of how to get that information (such as decoding a byte array, reading the bytes, looking for patterns, etc):

I've been working with XML and to me there are clear benefits. The main one is that it's human readable. So you can sometimes find information without knowing about the structure of a file format.

If we were to write the specification for JPEG now in XML it might look like this:

<s:Image xmlns="www.w3c.org" width="1000" height="600" bits="8">
      <s:BitmapData>0F8320100830F0A0230B09CC0...</s:BitmapData>
</s:Image>

My question is, if XML was created around the same time as the JPG and PNG and file size was not as much an issue (bandwidth was a huge issue in the early days of the internet) would they have used XML to save JPEG information or would they have chosen to write data to a byte array using specific markers for storing information?

What would you do if it was your choice?

Answer 1

It is certainly possible that the JPEG and PNG people might have used XML-based headers if file size were not an issue. Using text would make it more obvious what all the fields mean and so forth.

But for storing the actual file data, almost certainly not. The data elements of JPEG/PNG images is a sequence of bytes, not XML elements that explain what that data means. So storing that as characters, even in base64 or some such, is simply pointless. It makes processing take longer than needed (since you have to do a lot of text-to-binary conversion), and all to nobody's benefit. It wouldn't make the data portion easier to understand; you'd still have to look up the actual compression algorithm and such.

Indeed, processing time would probably be the reason to avoid even XML headers. Sure, text parsing isn't that expensive, but it's not nearly as cheap as blasting some data into memory, casting a pointer, and reading a struct.

Answer 2

Byte arrays are human readable. I've been reading them since I was 10 and I've been human for longer than that. This was before I even had a fancy hex editor. Peek and Poke commands are old friends.

What an xml document gives you isn't human readability. It gives you a reasonably predictable encoding (ASCII, UTF-8, etc) that lets you use notepad or vi as your editor, and it gives you meta information right there in the file.

The hex editor image you posted of the byte array has lots of lovely meta information. This meta information you added, using some paint program, simply isn't in the byte array. You only know that info because you know this is a JPEG and you know how JPEG's are structured. You had to go look that up in a specification document. Sometimes programmers aren't in a position to publish and distribute meta information separately from the data file. My tick tack toe games' file format is unlikely to show up in Wikipedia any time soon.

Being able to leave out meta information and being able to use every value of a byte means byte array's are smaller. But it's a O(0.6n) kind of smaller. If that really is an issue you can always use compression. Seriously, size is really not the primary issue.

Look at excel spread sheets for example. Change their extension to .zip and extract them and you find a ton of xml files hiding underneath.

A byte array file can be just as dynamically structured as an xml or json file can be. But unless the markers of that dynamic structure are published somewhere, good luck figuring out the structure.

The JPEG structure is already well known so converting it to XML so you can distribute that structure as meta information is solving a non problem.

I use XML or JSON when I don't feel like publishing a spec to define my structure. I use byte arrays when I don't care if anyone else ever figures out my structure or when I figure my structure is something I can get published.

Answer 3

Two things:

1. The benefits of your scheme are small and questionable: yes I can parse and extract metadata a bit more easily. But you only really need one C library to parse an image format and then it's a solved problem.

2. Even with modern bandwidth image size is a huge concern. Think about browsing Facebook from a mobile phone -- there will be hundreds of images being constantly loaded. You don't want it to take way longer and eat through your data plan because the developers found this XML format moderately easier to work with.

Just to stress what a difference there is: by encoding the data as a hex string you are using 4 times as much space. In many contexts that's completely unacceptable and it's really not buying you much.

Answer 4

You example XML isn't even a good example, as you're still using a byte array. A 'true' XML for an image, to contrast with byte arrays, would be something like:

<s:Image xmlns="www.w3c.org" width="1000" height="600" bits="8">
   <s:pixels>
      <s:row>
         <pixel>000000<pixel>
         <pixel>000001<pixel>
         <pixel>000002<pixel>
      <s:row>
      <s:row>
         <pixel>000010<pixel>
         <pixel>000011<pixel>
         <pixel>000012<pixel>
      <s:row>
      ...
   </s:pixels>
</s:Image>

Or some similar unholy creation. Even if disk space is cheap, that is a pretty serious cost of lots of angle brackets. Now sure, you could .zip or compress your XML file, but then you're introducing a tooling requirement to see what's up with the file, so at that point you might as well instead make the file format more sensible and have tooling move the data in the right direction.

To elaborate on some of Nicol Bolas' answer, with an example of image formats, or anything involving compression, you instantly lose any sort of human readable aspect, as the better the compression, the more incomprehensible the result. The format I had in my example is a bitmap. A jpeg is not going to be human readable in any format, as it's data is many steps removed from what the pixel values are by some serious math.

An additional behavior you can do with byte arrays, but is impossible with XML, is lossy data transfers. Common in signal processing applications, sometimes it's not important to, and is required to be able to, read and utilize a stream of data even when not every bit is correctly transfered. Streaming video is one such application. In the case of an XML file this is not acceptable as missing a single bit where you expect a closing ">" and your whole data structure will fail to parse.