Saving and reading files in user specific application data in C or C++

Question

Most useful or reasonably complex applications need to save data such as user settings or saved games or browser history. I have been working on applications and games in C or C++ but I am not sure which of several ways to access files to use.

First the preferred location to store such data is queried. On Linux this would be the app name appended to /. appended to getenv("HOME") or getpwuid(getuid())->pw_dir. A more advanced application would use a portable library or framework interface to query the location without manually checking the operating system and using the appropriate method. These concerns are mostly covered in Best way to save application settings.

The next part is accessing the files of various names within the directory. These are a few ways to do this:

• Simply concatenating /Name and storing this as a file path (either manually using memcpy() in C or using std::filesystem::path in C++).
• Opening and maintaining a file descriptor to the application directory and then using relative filesystem calls such as openat().
• Using chdir() on the application directory at application launch followed by open() when the file is needed.

What are the implications of each of these ways including their advantages or drawbacks? More specifically:

• The first option uses string manipulation so is less clean in my opinion but does not necessarily rely on POSIX-specific functions and therefore is probably more portable, but is the OS having to traverse the directory trees from the beginning of an absolute path going to be slower than the other options?
• How will keeping extra file descriptors open as in the 2nd option affect memory usage or performance? How will this affect stability, as having too many open file descriptors can prevent opening new ones, is this likely going to be an issue?
• How would each of these options work out if I have subdirectories inside the app data directory such as MyGame/Saves/<Save Name>? The 3rd option will not work here well as I cannot chdir() to a new directory temporarily without losing previous directory passed to chdir() but is there a way around this?

Answer 1

I would heavily avoid any solution which uses chdir, since it is way too errorprone. It might interfer with other places in a codebase assuming the current directory wasn't changed (or worse - change it by themselves). It may also cause astonishing side effects when a program grows over time and one changes the order in which certain code sections are executed. The only cases where using chdir makes sense is when you are 100% sure it is used exclusively at the very beginning at the startup phase of your program, in a single thread, before any other file operation takes place, and when you are sure the original current directory is restored afterwards.

This leaves you between using paths in strings or using file descriptors and openat. But as you already noted by yourself, the Linux/Unix call openat isn't portable between major OS platforms (to be fair, there are similar alternatives for Windows ) and it requires an extra file descriptor. The latter might be negligible for a single directory, but when you start using multiple file descriptors in parallel, and you expect to have many files, you might reach the OS limit (which AFAIK is 1024 in contemporary Linux systems, 512 under Windows, so it is indeed a somewhat limited resource). The only real advantage of openat over strings is that one can open the descriptor, then move/rename the directory while the descriptor is still open. If that is a scenario you expect to happen, you should use openat.

The first option uses string manipulation so is less clean in my opinion

Well, "less clean" sounds more like a superstitious feeling than some factual argument. To me, using strings (or maybe a lightweight encapsulation of strings for file paths) looks like a very clean solution as long as one uses standard functions to combine separate parts of a path. In modern C++, this should not be an issue. How to do this precisely depends on the framework and C++ version you are using. If you use C and/or don't have modern libraries/frameworks at hand, you will probably have to reimplement such functions by yourself.

but does not necessarily rely on POSIX-specific functions and therefore is probably more portable, but is the OS having to traverse the directory trees from the beginning of an absolute path going to be slower than the other options?

Maybe it is slower, but does it really matter for your specific real world application? Don't forget traversing the tree is only part of the operation, usually the most time consuming part is reading or writing the file. If you really run into a situation where you notice a measurable performance impact, and you know for sure traversing the file system tree is the bottleneck, then it is time to optimize - and using file descriptors instead of strings might be only one possible optimization (and may not bring the improvement you are looking for). Simply using fewer, but larger files instead of multiple small ones, or some lightweight file-based database can be more effective than trying to optimize file path access. It makes IMHO no sense to overthink this beforehand. My experience is that prematurely micro-optimizing code, especially code which hasn't been written, run and tested yet is a waste of time.

So in short, by "default", I would go with strings (or some string-based file path abstractions of your framework) plus standard functions for combining parts of a path, and switch only to something different if I expect the directories to change the location during a program's run. If I really see a performance issue where I suspect using file descriptors for intermediate directories could bring some benefit, I might write a performance test first, for the specific environment, OS file system and usage pattern of the specific application. And only when that test shows some notable improvement, then I would consider to use file descriptors (but not "just in case").