Skip to main content

Questions tagged [file-formats]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
1 answer
147 views

Selecting the endianness of data in files generated by an embedded system

I have an embedded system running on a little-endian Cortex-M3. This system is able to accept packets from the network and reply to them. Now, I would like the system to start generating files. These ...
9a3eedi's user avatar
  • 2,109
-1 votes
1 answer
133 views

Why do HTML and JS feel slow on smartphones? What could be changed to make them faster? [closed]

HTML rendering and javascript engines are very optimized, so why do they feel slow on smartphones? It seems that HTML's low performance justified the creation of mobiles "apps" which are ...
jokoon's user avatar
  • 2,242
1 vote
3 answers
209 views

Using source code instead of XML/JSON or other custom serialization schemes and binary file formats

For a while now I have been toying with the idea of using source code as a file storage format. My question: How to support format version changes? (loading of older files with structural differences) ...
Reto Höhener's user avatar
-3 votes
1 answer
96 views

Is it possible to extend gifs to support audio? [closed]

Can the file format gif be extended to support audio as well? I was looking into gifs that support audio and the main gif sites with "audio gifs" are just looping videos. The problem is how ...
wanna_coder101's user avatar
13 votes
8 answers
4k views

In what configuration file format do regular expressions not need escaping?

I want a configuration file for a .NET program. This file is to configure pairs of regular expressions. The regular expressions belong within a hierarchy of sections. Section1 SubsectionA ...
ChrisW's user avatar
  • 3,417
-2 votes
1 answer
93 views

Choosing a MagicNumber or Signature for a Binary File Format

I'm defining some binary formats to encode some information and dump it to a file. I want to place a MagicNumber/Signature at the beginning of files encoded in my format to be able to tell them apart ...
Blue Nebula's user avatar
0 votes
1 answer
613 views

Best file formats for ML training

Hi I'm bulding an ML pipeline with PyTorch to support various tasks and am looking for some advice on efficient ways to store processed data. The main frame work was 3 layers [data prep] -> [data ...
user2757771's user avatar
-1 votes
2 answers
804 views

How to handle multiple versions of binary file format

I'm writing a reader for the custom binary file format used in one game. The format has multiple versions for example: struct FileHeader { ushort version; uint fieldX; // version 1 uint ...
Konrad's user avatar
  • 1,559
0 votes
1 answer
1k views

Getting file format by checking file header [closed]

I'm writing a program, part of which consists of determining if a given file is a PNG. Knowing that a file doesn't have to be named with its respective filename extension to be of a certain type, I ...
Areg Sarvazyan's user avatar
0 votes
2 answers
92 views

What are formats to store geographic maps for a robot to travel point A to point B

Very beginner. Need to compute route from A to B for a robot. i would like to know how to store area map and compute the route. I can compute shortest path etc using algorithms e.g. dijkstra. What ...
Adams's user avatar
  • 47
1 vote
2 answers
493 views

Design Pattern to extract arbitrary field from arbitrary file format

Lets say I have multiple file types: .json, .csv ... etc These file types come in different formats: Second json structure Extra column added to csv etc. I need to extract fields from these files;...
Christian Gabor's user avatar
1 vote
2 answers
388 views

How a standard video file is structured under the hood

I am new to looking into video file formats, and am wondering what it would take to parse one. To do that, I first would need to understand what the format looks like, so that's what this question is ...
Lance's user avatar
  • 2,615
47 votes
4 answers
39k views

Should UTF-8 CSV files contain a BOM (byte order mark)?

Our line-of-business software allows the user to save certain data as CSV. Since there are a lot of different formats (all called "CSV") in use in the wild, we are tying to decide what the &...
Heinzi's user avatar
  • 9,798
9 votes
2 answers
2k views

SQLite database as a data interchange format?

Is it sensible/reasonable to consider using an SQLite database file as a data interchange format? Are there any obvious gothchas or reasons not to do this? Example. Say I have a master SQL Server ...
redcalx's user avatar
  • 365
1 vote
1 answer
2k views

Fast storage format for huge point clouds (fast read/write)

I am working with huge point clouds (1 billion points). I need to process and display them but because of the size of the data, I can't have it all in memory at the same time. First of all, I am not ...
Ebatsin's user avatar
  • 111
1 vote
0 answers
232 views

Best practice to handle input and output with several csv-like files

I am currently writing an application with the following structure: input: the equivalent of an excel workbook, i.e. a few tables with different headers and a few scalar values. They represent ...
Andreas's user avatar
  • 19
-1 votes
1 answer
283 views

How do different file types generally store data?

I am working on a project, for which I want to create my own file format to store large amounts of data. I am trying to decide how that file format should be built to be as efficient as possible. The ...
zomnombom's user avatar
  • 101
1 vote
6 answers
1k views

What are the benefits of storing data contiguously?

I am designing an application file format which will store chunks of user data, ranging from a few bytes to a few gigabytes - median size probably in the 10MB - 30MB range. I have the option of ...
jl6's user avatar
  • 423
3 votes
4 answers
1k views

Would you use Byte Arrays or XML if size was not an issue?

In my basic understanding of ByteArrays the benefit is that it is smaller in file size. The down size of a ByteArray is that for any given format you have to know the about the file format to get ...
1.21 gigawatts's user avatar
0 votes
1 answer
831 views

best way to export binary data with additional text attributes to a stream

I would like to create a command line API in a Java application to export a binary blob to stdout. I would additionally like to export certain attributes about the blob, ideally in a non-binary format....
Reinstate Monica's user avatar
1 vote
3 answers
528 views

What is the simplest translation file format for a user to edit and for a program to parse? [closed]

I have just started developing a small web app for Arabic speakers, and I will need to ask a co-worker (not a programmer) to help me translate all of the labels and documentation into Arabic at some ...
Dan Ross's user avatar
  • 369
2 votes
3 answers
1k views

What metadada to save and whether to include it inside the binary file?

I'm wondering if there are any general guidelines or best practices regarding when to split data into a metadata format, as oppose to directly embedding it within the data. (Specific example below). ...
smg's user avatar
  • 135
21 votes
2 answers
10k views

What is the current status of software support for JPEG-2000?

The general recommendation to record original scanned images used to be "use TIFF". But programmers need evolution of format for "evolution of software", and I need to evolve my system to change from ...
Peter Krauss's user avatar