Optimal storage space calculation for name, address, phone, fax and email data

Question

Given the following assumptions:

• The amount of available data storage is highly limited
• All data is international
• As much storage as possible must be preserved since it will be used for something else

Is there any widely accepted standard or best practice which defines how much storage should be allocated for those items?

Note: I'm not looking for assumptions but for facts like "97.5% of all names fit into 70 bytes using UTF-8" and "An international address must at least contain field1, field2 and field3 of size1, size2 and size3".

Edit: I'm looking for the fields/lengths to use in the user interface. Not a technique to store it efficiently.

Answer 1

Coming at it from a slightly different perspective, since your goal is to store things as compactly as possible, avoid using fixed width fields.

To pull from your own example:

97.5% of all names fit into 70 bytes using UTF-8

You shouldn't have a fixed 70 byte allocation for the name.

Instead, have a fixed allocation for the entire record - say, 200 bytes - then store all the data in that one record with a delimiter between fields.

For example:
Title/Given/Middle/Family/KnownAs/Born
Mr/William/Henry/Gates/1955-10-28

This is a classic space/time tradeoff as searching such a structure will be slower.

One key advantage of this structure is that it allows for some fields to be unusually long without compromising storage. So, if someone's street address happened to require 70 characters, or if their full name took 90, you can accomodate this as long as the same person didn't have both.

Answer 2

If the space is limited, then the overhead of skipping elements one-by-one wouldn't be so important.

Thus, you can construct a tree, made of lists of length-prefixed elements. You can encode the lengths as single bytes, and for records meant to be larger than what this allows, you can shift the value to the left (that wouldn't waste because it's compensated by the smaller field).

Also, if you want to simplify your code further, it might make sense to embed a flag in the size field indicating if the record is raw or a list. That would make trivial to parse the whole data set (either incrementally or on a single pass).

That would be much more efficient than the fixed-length statistical approach.

Plus, you might want to consider applying some form of compression to the whole thing to save some more space.