Is there a standardised way of adding custom metadata to image and other filetypes?

Question

What's a good way of storing and referencing files of different types (csv, pdf, jpg, html), each of which needs my own custom metadata? The metadata stores details of the content and how it was created. Ideally I could add metadata to the file itself, for CSV that's going to be #comments. PDF similarly supports it's own Metadata, but for jpg the EXIF format seems too limited and XMP needs it's own separate file. HTML - hmm not sure of the best approach.

This needs to work on Windows and Linux, code is .Net Core initially but maybe node.js later.

Answer 1

For WAV, AIFF, PNG and other IFF-like file formats, there is a standard way: define additional chunk types for your additional metadata.

For the other file formats in question, you can use format-specific extensions as you summarized.

In HTML that could be <meta> tags, or <!-- comment --> tags containing structured text, or custom attribute=value pairs. If the custom metadata is binary, you can encode it with base64 to include it in HTML.

All these format-specific extensions will require custom reader/writer code.

Alternatively, you could store the additional metadata in a uniform way outside the files. That could be in a database, a sidecar file per data file, a metadata file per directory, or other place.

If the data files are all inside a zip file, you can use zip comment fields.

If the files are all on Unix (including macOS) and Linux, you can use extended file attributes (xattr).

Some file systems support a "file system fork" for each file. On Windows NTFS it's called an "alternate data streams (ADS)".

The design choices have the obvious tradeoffs in which operations preserve or drop the custom metadata.