Skip to main content
Share Your Experience: Take the 2024 Developer Survey
146 votes
Accepted

Should temporary files be saved to /tmp or the current working directory?

Temporary files have to be stored into the operating system temporary directory for several reasons: The operating system makes it very easy to create those files while ensuring that their names ...
Arseni Mourzenko's user avatar
59 votes
Accepted

What is the advantage of log file rotation?

"I have never had any issues with large files" is not synonymous with "there is never a problem with large files". Your lack of experience with problems does not prove that there ...
Flater's user avatar
  • 51.7k
33 votes

Should temporary files be saved to /tmp or the current working directory?

Should I insist saving to /tmp is the right approach and defend for any failure as "working as intended" (ie. ask your admin for proper permission access)? There are standards for this, and the best ...
Blrfl's user avatar
  • 20.4k
29 votes

What is the advantage of log file rotation?

Log file rotation is not only related to technical issues with the file itself. It's also related with operational considerations, like: being able to incrementally backup the logs (especially in ...
Christophe's user avatar
  • 77.9k
22 votes
Accepted

Why would anyone use multipart/form-data for mixed data and file transfers?

multipart/form-data is a construct created for HTML forms. As you've discovered the positive of multipart/form-data is the transfer size is closer to the size of the object being transferred--where in ...
Samuel's user avatar
  • 9,197
18 votes
Accepted

Java - Processing a large file concurrently

The most likely efficient way to do this is: Have a single thread that reads the input file. Harddisks are at their fastest when reading sequentially. Do not read it into memory all at once! That is ...
Michael Borgwardt's user avatar
18 votes
Accepted

Using flat files vs database/API as a transport between a frontend and backend

Switching to a solution involving databases or the queuing systems mentioned by Ewan would create dependency on a new, complex system in both backend and frontend introduce unnecessary complexity and ...
DarthGizka's user avatar
16 votes
Accepted

Is it bad practice to read large file in constructor?

When seeing a class with a constructor signature like EnglishWordsListGenerator(const std::string &wordFileName) I think it is pretty obvious that this constructor will read the given file (and ...
Doc Brown's user avatar
  • 209k
15 votes

What is the advantage of log file rotation?

The other answers already provided very useful reasons, but there is one more I'm surprised I haven't seen outside of a comment yet: To be able to delete old log files to free disk space A very busy ...
Syndic's user avatar
  • 312
14 votes
Accepted

Forward Slash vs Back Slash vs File.separator in Java

No, there isn't. Slashes work everywhere, backslashes work only on Windows and are a pain to type and read. They are used only by people who mistakenly think they have to use them. But of course, you ...
Kilian Foth's user avatar
11 votes

Using flat files vs database/API as a transport between a frontend and backend

I don't think either solution is inheritly a bad practice, so answering which is the best practice may be difficult. I don't believe the YAGNI principal applies here if you're dealing with scale. "...
user161778's user avatar
11 votes
Accepted

Base64 in Json: Is it a Good Idea for Rest API?

Base64 is a safe encoding for JSON. The downside to using Base64 is that it's quite verbose. It can add significant overhead to your file upload which means longer upload and more bandwidth used. If ...
Samuel's user avatar
  • 9,197
10 votes

Should temporary files be saved to /tmp or the current working directory?

The previous answers, although correct, aren't valid for most large scale computer clusters. Computer clusters not always follow the standard conventions for machines, usually for good reasons, and ...
Davidmh's user avatar
  • 230
10 votes
Accepted

Storing images in base64 vs binary, in cloud

With base64, you get a 33.3% overhead because of the 4:3 ratio. This means that for a 3 MB image, you will pay 4 MB of storage, and possibly, depending on the pricing model, 32 Mbps of network ...
Arseni Mourzenko's user avatar
9 votes

Should temporary files be saved to /tmp or the current working directory?

The temp-file-directory is highly operating system/environment dependant. For example a web-servers-temp dir is seperate from the os-temp-dir for security reasons. Under ms-windows every user has ...
k3b's user avatar
  • 7,526
9 votes

How is a software able to read a network file faster than it appears to be possible?

A network of 100 Mbps (mega bits per seconds) conveys 12,5 MB per seconds, including payload and protocol overhead. A file of 165 MB needs at least 13,5 seconds (In fact, it would require slightly ...
Christophe's user avatar
  • 77.9k
9 votes

Load and process (compressed) data from filesystem in the blink of an eye

profiling measurements It sounds like you do not yet know where the CPU cycles go, or, more importantly, the sources of I/O + network delay. I didn't see any numeric figures, let alone 98th percentile ...
J_H's user avatar
  • 7,156
8 votes

What is the advantage of log file rotation?

never had any issues with large files and cannot think of reasons why we might set an arbitrary limit Every system I have ever ssh'd into has had limited budget and limited attached storage. If you ...
J_H's user avatar
  • 7,156
7 votes

Java - Processing a large file concurrently

Let's start with some basic arithmetic. (* 24 60 60) 86400 This means that there are 86400 seconds in one day. (/ 100e6 86400) 1157.4074074074074 What this means is that to process 100 million ...
John R. Strohm's user avatar
7 votes

Is it bad practice to read large file in constructor?

If you are new to C++, it may be desirable to shy away from doing big work in a constructor. Constructors are wedded tightly to the way the language behaves, and you really don't want the constructor ...
Cort Ammon's user avatar
  • 11.2k
6 votes

PHP file_put_contents File Locking

I know this is ages old, but in case someone runs into this. IMHO the way to go about it is like this: 1) Open the original file (e.g. original.txt) using file_get_contents('original.txt'). 2) Make ...
Dom's user avatar
  • 161
6 votes

First time having users - how to deal with backwards compatibility?

A good start would be to resist the urge to keep changing the file format. If you really must, think about backward compatibility. Adding new features to an existing file format doesn't stop you ...
Simon B's user avatar
  • 9,633
6 votes

Is there a standardised way of adding custom metadata to image and other filetypes?

For WAV, AIFF, PNG and other IFF-like file formats, there is a standard way: define additional chunk types for your additional metadata. For the other file formats in question, you can use format-...
Jerry101's user avatar
  • 5,427
6 votes
Accepted

Constantly writing a JSON file

The last time I had a requirement like that, the process looked like When starting a new file, write the characters [\n to it to open a JSON array. When appending to an existing file, seek to the end....
Bart van Ingen Schenau's user avatar
6 votes
Accepted

Checking if a file exists before writing. Always avoid, or sensible with the right use case?

When opening a file (regardless if it is opened for reading or writing), one has to do proper exception / error handling for the specific "open" statement (as well as for any further I/O). ...
Doc Brown's user avatar
  • 209k
6 votes

Allow-Rendering-Prevent-Download Architecture

Fundamentally, you are trying to solve an impossible problem - there is no difference between "rendering a file in the client's browser" and "download the file". They both involve ...
Philip Kendall's user avatar
6 votes

What is the advantage of log file rotation?

I worked on a bug last year where an action in production was suddenly failing due to a timeout. Fast, near instantaneous in dev and QA, failing in production. Took weeks of on again off again work ...
jmoreno's user avatar
  • 11k
5 votes

Using flat files vs database/API as a transport between a frontend and backend

While the good 'ol save a file and copy it to a done dir is a staple of many communication layers esp. with older main frame systems and the like. The 'anti' guys do have a point; in that it has many ...
Ewan's user avatar
  • 76.4k
5 votes

Difference between '\n' and '\r\n'

Here is an answer from the best source - Microsoft. Why is the line terminator CR+LF? This protocol dates back to the days of teletypewriters. CR stands for "carriage return" - the CR ...
Ondra Žižka's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible