Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

The tag has no usage guidance.

0
votes
2answers
101 views

Design Pattern to extract arbitrary field from arbitrary file format

Lets say I have multiple file types: .json, .csv ... etc These file types come in different formats: Second json structure Extra column added to csv etc. I need to extract fields from these files;...
0
votes
2answers
185 views

How a standard video file is structured under the hood

I am new to looking into video file formats, and am wondering what it would take to parse one. To do that, I first would need to understand what the format looks like, so that's what this question is ...
34
votes
2answers
4k views

Should UTF-8 CSV files contain a BOM (byte order mark)?

Our line-of-business software allows the user to save certain data as CSV. Since there are a lot of different formats (all called "CSV") in use in the wild, we are tying to decide what the "default ...
2
votes
2answers
174 views

SQLite database as a data interchange format?

Is it sensible/reasonable to consider using an SQLite database file as a data interchange format? Are there any obvious gothchas or reasons not to do this? Example. Say I have a master SQL Server ...
0
votes
1answer
177 views

Fast storage format for huge point clouds (fast read/write)

I am working with huge point clouds (1 billion points). I need to process and display them but because of the size of the data, I can't have it all in memory at the same time. First of all, I am not ...
1
vote
0answers
67 views

Best practice to handle input and output with several csv-like files

I am currently writing an application with the following structure: input: the equivalent of an excel workbook, i.e. a few tables with different headers and a few scalar values. They represent ...
0
votes
0answers
63 views

Implementing URLConnection#getContentType() for non-MIME connections

I've been messing around with networking in Java and have been implementing various protocols as practice. One of the protocols I implemented was Gopher, a directory-style resource browser that ...
-1
votes
1answer
127 views

How do different file types generally store data?

I am working on a project, for which I want to create my own file format to store large amounts of data. I am trying to decide how that file format should be built to be as efficient as possible. The ...
0
votes
5answers
254 views

What are the benefits of storing data contiguously?

I am designing an application file format which will store chunks of user data, ranging from a few bytes to a few gigabytes - median size probably in the 10MB - 30MB range. I have the option of ...
3
votes
4answers
281 views

Would you use Byte Arrays or XML if size was not an issue?

In my basic understanding of ByteArrays the benefit is that it is smaller in file size. The down size of a ByteArray is that for any given format you have to know the about the file format to get ...
0
votes
1answer
477 views

best way to export binary data with additional text attributes to a stream

I would like to create a command line API in a Java application to export a binary blob to stdout. I would additionally like to export certain attributes about the blob, ideally in a non-binary format....
1
vote
3answers
152 views

What is the simplest translation file format for a user to edit and for a program to parse? [closed]

I have just started developing a small web app for Arabic speakers, and I will need to ask a co-worker (not a programmer) to help me translate all of the labels and documentation into Arabic at some ...
2
votes
3answers
191 views

What metadada to save and whether to include it inside the binary file?

I'm wondering if there are any general guidelines or best practices regarding when to split data into a metadata format, as oppose to directly embedding it within the data. (Specific example below). ...
20
votes
2answers
9k views

What is the current status of software support for JPEG-2000?

The general recommendation to record original scanned images used to be "use TIFF". But programmers need evolution of format for "evolution of software", and I need to evolve my system to change from ...