Best approach to persist settings in files

Question

I am in the process of migrating the way an app makes persistence of its settings.

It wrote and read data from registry; now it is desired it load/save XML files on the C:\AppData\MyApp\Version folder for several reasons.

It had lots of folders on the registry.

1st question: Whats is the best approach: A mega file having the whole configuration or several files?

2nd question: The organization of the registry folders, values and keys is a huge mess and not matching the source structure. There is much inter-crossing between the registry folders organization and classes.

There are some values put to / got from registry SURGICALLY. I mean parts of code that only read/write only one entry. Given I am working with files, I do not know a better manner to do it besides load and and save the whole files! Can anyone suggest better approaches?

3rd question: More documentation on the subject, please!

Answer 1

Have you tried SQLite? It's a self contained embedded lightweight SQL database. A lot of applications use it to store user settings and other persistence use cases in Desktop Apps - all the major Browsers, OSes, Skype and more.

Of course this would mean your user settings won't be "Human Readable" as such. Not as much as XML anyway. Is that a requirement and is that why you are using XML?

You can even just transport the SQLite file, incase you wanted users to be able to port the app (user settings intact) to another machine.

The problem you'll face as you rightly put it, is that in any text based serialisation / persistence scheme (JSON, XML) if you are auto saving User Settings you will have to WRITE the entire file on save. A database, by virtue of indexes and it's internal storage structure can avoid this cost.

Though I come from a multi platform background, I've seen that the trend for modern desktop apps on windows is to avoid the registry as much as possible. Mainly because the uninstall scenario is such a pain for users who struggle with apps that don't clean ALL their entries in the windows registry.

To summarise, based on readability, portability and performance requirements as detailed above, you can choose One large XML file, Several Smaller ones or One SQLite database.

Answer 2

Accepting that having persistence is a good thing, and that the amount of data needed for that persistence can vary from almost nothing (a single character or word), to large quantities of data, with extreme complexities, there certainly is a need to be able to make changes as the system grows.

Depending on that amount of, and complexity of the persistence data, one or more files using XML, JSON, or similar things often work, until the volume of data become 'too large'.

While deciding between "a mega file having the whole configuration or several files", also consider the type of data: Is it primarily highly structured data, or not structured.

If highly structured, then a solution such as SQLite is a good option. On data is in the DB, and indexed etc., most issues you mention will be easy to resolve.

I have a similar issue: rather than trying to provide persistence for a single app, my issue is a large quantity of data that has been added to, and read by, several different apps. In spite of trying various ways to combine all of these, such that all entries that pertain to a specific object are combined, then eliminating the duplicates, have failed. Usually because the apps were designed to work with much smaller blobs of data.

SQLite does not work for this, because the data is not well structured: I keep finding new items that should have been in the data dictionary.

I recently was introduced to MongoDB, a NoSQL type of database. For my needs, since all my files are such that I can extract the desired data and create JSON documents, I am hoping that MongoDB (or a similar type of DB) will be able to do the combining, sorting, and De-duplication I need.

If you can create JSON documents (or some equivalent) with the data you currently have and massage that with MongoDB or other NoSQL program, you should be able to find the answers, as Robert Harvey said above.

If you end up with well organized data, SQLite could be a great solution, or you may find that a simple XML file may work.