0

What's a good way of storing and referencing files of different types (csv, pdf, jpg, html), each of which needs my own custom metadata? The metadata stores details of the content and how it was created. Ideally I could add metadata to the file itself, for CSV that's going to be #comments. PDF similarly supports it's own Metadata, but for jpg the EXIF format seems too limited and XMP needs it's own separate file. HTML - hmm not sure of the best approach.

This needs to work on Windows and Linux, code is .Net Core initially but maybe node.js later.

2
  • 1
    Storing custom metadata inside an image file looks like to have the advantage of keeping the data and the file together - at a first glance. Unfortunately, it has the drawback that this metadata gets lost as soon as one uses some image processing tool which does not anything about the custom extensions. So you need to think about your actual use cases - or even better, describe them in your question, that would allow the community to give you a more specific answer. – Doc Brown Dec 17 '20 at 6:29
  • Xmp can be embedded in many file types. – Kris Van Bael Dec 17 '20 at 7:25
3

For WAV, AIFF, PNG and other IFF-like file formats, there is a standard way: define additional chunk types for your additional metadata.

For the other file formats in question, you can use format-specific extensions as you summarized.

In HTML that could be <meta> tags, or <!-- comment --> tags containing structured text, or custom attribute=value pairs. If the custom metadata is binary, you can encode it with base64 to include it in HTML.

All these format-specific extensions will require custom reader/writer code.

Alternatively, you could store the additional metadata in a uniform way outside the files. That could be in a database, a sidecar file per data file, a metadata file per directory, or other place.

If the data files are all inside a zip file, you can use zip comment fields.

If the files are all on Unix (including macOS) and Linux, you can use extended file attributes (xattr).

Some file systems support a "file system fork" for each file. On Windows NTFS it's called an "alternate data streams (ADS)".

The design choices have the obvious tradeoffs in which operations preserve or drop the custom metadata.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.