165 votes
Accepted

Why do most log files use plain text rather than a binary format?

systemd famously stores its log files in binary format. The main issues I have heard with it are: if the log gets corrupted it's hard to recover as it needs specialist tooling they are not human ...
Alex's user avatar
  • 3,922
138 votes
Accepted

Does storing plain text data take up less space than storing the equivalent message in binary?

Plaintext is binary. When you write an H to a hard drive, the write head doesn't carve two vertical lines and a horizontal line into the platter, it magnetically encodes the bits 010010001 into the ...
8bittree's user avatar
  • 5,656
105 votes
Accepted

Would having 4 states per "bit" rather than 2 mean twice the storage space?

The word you are looking for is not "bit" but "symbol." "Symbol" is the word used to describe the process of mapping hardware signals (such as voltages or magnetic patterns) into logical bits. If a ...
Cort Ammon's user avatar
91 votes

Why do most log files use plain text rather than a binary format?

Why do most log files use plain text rather than a binary format? Search for the word "text" in the Unix philosophy Wikipedia article, for example you'll find statements like: McIlroy, then ...
ChrisW's user avatar
  • 3,417
54 votes
Accepted

What is the size of the number 65535 in bytes?

TL;DR The key takeaway here is that there is a world of difference between the number 65535 and a piece of text which represents the digits '6', '5', '5', '3' and '5'. It may look the same to you when ...
Flater's user avatar
  • 49.8k
49 votes

Why do most log files use plain text rather than a binary format?

There are a lot of debatable presumptions here. Logging has been an integral part of (almost) every job I've had. It is essential if you want any sort of visibility on the health of your ...
Robert Harvey's user avatar
36 votes

Why do most log files use plain text rather than a binary format?

Log files are a critical part of any serious application: if the logging in the app is any good, then they let you see which key events have happened and when; what errors have occurred; and general ...
SusanW's user avatar
  • 1,035
21 votes
Accepted

How are variables stored in a language compiler or interpreter?

Interpreter An intepreter will work about the way you guessed. In a simple model, it will maintain one dictionary with the variable names as dictionary keys and the variable values as dictionary ...
Ralf Kleberhoff's user avatar
21 votes

Would having 4 states per "bit" rather than 2 mean twice the storage space?

One quarternary memory cell can store exactly as much information as 2 binary memory cells: Quaternary Binary 0 00 1 01 2 10 3 11 So if you have same number of ...
hyde's user avatar
  • 3,754
20 votes

Does storing plain text data take up less space than storing the equivalent message in binary?

I find this a great fun thing to think about. Binary is not 1s and 0s in the way you talk about it. Imagine there is a quantity, I can tell you what quantity it is in many different ways: Nine in ...
TessellatingHeckler's user avatar
16 votes
Accepted

Group set of commands as atomic transactions (C++)

In practice, such atomic transactions do not exist. Instead, you can try to make all operations idempotent so that they can be safely retried. Idempotence is particularly important for network ...
amon's user avatar
  • 134k
14 votes

Does storing plain text data take up less space than storing the equivalent message in binary?

Does storing plain text data take up less space than storing the equivalent message in binary? No, never. Your computer already stores the plain text data in the equivalent binary representation. ...
Telastyn's user avatar
  • 109k
14 votes

Is it ineffecient to store data as json in plain text?

Is it less efficient to store a small bit of data like your example encoded in a string than as binary? Yes. How much less? Not enough to care. Is it less efficient to store thousands of such records ...
candied_orange's user avatar
12 votes
Accepted

Binary data formats, how to make ensure you can read different format versions?

It seems to me that the simplest solution is to make your version header unambiguous and make sure that the old format can never look like it has a format header, you simply look for it. If it's not ...
JimmyJames's user avatar
  • 27.3k
11 votes

Why do or should programmers save data in text based formats like JSON or XML instead of binary?

JSON and XML are less efficient than pure binary, but what you aren't considering is that they (JSON/XML, and some other formats like YAML) are standardised, while binary isn't - different systems, ...
AIWalker's user avatar
  • 1,307
11 votes
Accepted

Why shouldn't I be using SQLITE instead of the INI format in all cases?

I think the problem is your definition of application settings. I think most people would designate your "partial and full downloads" as application state rather than application settings, ...
Ewan's user avatar
  • 75.6k
10 votes

How are variables stored in a language compiler or interpreter?

It depends on the implementation. For example, a C compiler might maintain a symbol table during compilation. This is a rich data structure that allows pushing and popping of scopes, since each ...
Erik Eidt's user avatar
  • 33.8k
9 votes

Why do most log files use plain text rather than a binary format?

An example of a somewhat binary log is wide-spread: the Windows event log. On the pro side, this allows log messages to be quite wordy (and thus hopefully helpful) at virtually no cost, possibly ...
Hagen von Eitzen's user avatar
9 votes

Should MySQL database tables only have a single unit of information per row?

Your coworker is actually right about this. If you're taking measurements from all your sensors at the same interval, and the set of sensors you're recording from remains constant or mostly constant, ...
Mason Wheeler's user avatar
9 votes

Would having 4 states per "bit" rather than 2 mean twice the storage space?

In basic theory, yes. In reality, no - because we don't actually store data in bits anyway (on HDDs). Cort Ammon covers the issues in data transmission very well. RAM, cache, and SSDs store data as ...
user3685427's user avatar
9 votes

Binary data formats, how to make ensure you can read different format versions?

Your old format starts with an 8-byte "size" element (probably a 64-bit long). If that's the file size in bytes (what I guess), it most probably never exceeded 100 GByte, meaning 2^37. So, if you ...
Ralf Kleberhoff's user avatar
8 votes

What is the preferable date-format used for designing APIs and storage

ISO8601. It's not just "preferable", it should be the only format you consider.
David Arno's user avatar
  • 39.3k
8 votes
Accepted

How to store external numeric IDs - as strings or as integers?

unless the datatype is specified then save it as the datatype it comes in. so if you have json: { "id" : 123 } Then you know you can 'safely' save it as a 'number' of unspecified precision if ...
Ewan's user avatar
  • 75.6k
8 votes
Accepted

Data storage for small application using XML

Storing data in textual files such as XML or JSON is perfectly fine, but has a couple of restrictions. Every time you need to read some data you first have to load the whole file, and for any change ...
amon's user avatar
  • 134k
8 votes

Should a SVG be converted to JSON to be stored in the database?

An other software engineer in my team told me to use JSON but couldn't substantiate why? Maybe it was a joke? I can't think of any benefit to storing SVG inside JSON. None. I would store SVG files......
Dan Wilson's user avatar
  • 3,103
8 votes
Accepted

What happens to the rest of memory address space if the data is less than 1 byte?

While all true and interesting in the context, Doc Brown's answer does not addresses the question the way I interpret it. The point is, in any microprocessor a byte is the smallest writable data unit. ...
Martin Maat's user avatar
  • 18.4k
8 votes

What is the size of the number 65535 in bytes?

There are likely several variables at play here, but 65535 is 5 characters. 1 byte per character = 5 bytes. Text editors don't have a notion of what a number is the same way that your code does, so ...
mmathis's user avatar
  • 5,478
7 votes
Accepted

Is it better to pull all data immediately or as needed?

It depends. If you know you will need all the data eventually, it is technically faster to download it all at once. However, if you can download a portion of the data and use it to keep the user ...
Dan Pichelman's user avatar
7 votes

Binary data formats, how to make ensure you can read different format versions?

The convention format version ID, everything else is good because it gives maximum flexibility for the format. In your case the version ID isn't right at the front but at a fixed offset. This slightly ...
amon's user avatar
  • 134k
7 votes

Should a SVG be converted to JSON to be stored in the database?

Neither. You should store the model the application uses to generate the svg. ie for a network diagram the various nodes and joins. That way when you change the svg generation code the old models ...
Ewan's user avatar
  • 75.6k

Only top scored, non community-wiki answers of a minimum length are eligible