Storing data in code

Question

A few times in my past I've wanted to store data in code. This would be data that seldom changes and is used in places where access to a database isn't possible, practical, or desirable. A small example would be storing a list of countries. For that you could do something like:

public class Country
{
    public string Code { get; set; }
    public string EnglishName {get;set;}
}

public static class CountryHelper
{
    public static List<Country> Countries = new List<Country>
        {
            new Country {Code = "AU", EnglishName = "Australia"},
            ...
            new Country {Code = "SE", EnglishName = "Sweden"},
            ...
        };

    public static Country GetByCode(string code)
    {
        return Countries.Single(c => c.Code == code);
    }
}

I've got away with doing this in the past because the data sets were relatively small and the objects pretty simple. Now I'm working with something that will have more complex objects (5 - 10 properties each, some properties being dictionaries) and around 200 objects total.

The data itself changes very rarely and when it does change it's really not even that important. So rolling it into the next release version is perfectly fine.

I'm planning to use T4 or ERB or some other templating solution to turn my data source into something that is stored statically in the assembly.

It seems my options are

1. Store the data in XML. Compile the XML file as an assembly resource. Load data as needed, store loaded data into a Dictionary for repeat-use performance.
2. Generate some kind of static object or objects that are initialized at startup.

I'm pretty sure I understand the performance implications of option 1. At least, my hunch is that it wouldn't have stellar performance.

As for option 2, I don't know what to do. I don't know enough about the internals of the .NET framework to know the best way to actually store this data in C# code and the best ways to initialize it. I poked around using .NET reflector to see how System.Globalization.CultureInfo.GetCulture(name) works, since this is actually a very similar workflow to what I want. Unfortunately that trail ended at an extern, so no hints there. Is initializing a static property with all the data, as in my example, the way to go? Or would it be better to create the objects on demand and then cache them, like this?

    private static readonly Dictionary<string, Country> Cache = new Dictionary<string,Country>(); 

    public static Country GetByCode(string code)
    {
        if (!Cache.ContainsKey(code))
            return Cache[code];

        return (Cache[code] = CreateCountry(code));
    }

    internal static Country CreateCountry(string code)
    {
        if (code == "AU")
            return new Country {Code = "AU", EnglishName = "Australia"};
        ...
        if (code == "SE")
            return new Country {Code = "SE", EnglishName = "Sweden"};
        ...
        throw new CountryNotFoundException();
    }

The advantage to creating them at once in a static member is that you can use LINQ or whatever else to look at all the objects and query them if you want. Though I suspect doing this has a startup performance penalty. I'm hoping someone has experience with this and can share their opinions!

Answer 1

I would go with option one. It's simple & easy to read. Someone else looking at your code will understand it straight away. It'll also be easier to update your XML data if needs be. (Plus you could let the user update them if you have a nice front end and stored the files separately)

Only optimise it if you need to - premature optimisation is evil :)

Answer 2

Given that these are essentially key-value pairs, I recommend storing these strings as a resource file embedded in your assembly at compile time. You can then read them out using a ResourceManger. Your code would look something like this:

private static class CountryHelper
{
    private static ResourceManager rm;

    static CountryHelper()
    {
        rm = new ResourceManager("Countries",  typeof(CountryHelper).Assembly);
    }

    public static Country GetByCode(string code)
    {
        string countryName = rm.GetString(code + ".EnglishName");
        return new Country { Code = code, EnglishName = countryName };
    }
}

To make it a bit more efficient you can load them all at once, like this:

private static class CountryHelper
{
    private static Dictionary<string, Country> countries;

    static CountryHelper()
    {
        ResourceManager rm = new ResourceManager("Countries",  typeof(CountryHelper).Assembly);
        string[] codes = rm.GetString("AllCodes").Split("|"); // AU|SE|... 
        countries = countryCodes.ToDictionary(c => c, c => CreateCountry(rm, c));
    }

    public static Country GetByCode(string code)
    {
        return countries[code];
    }

    private static Country CreateCountry(ResourceManager rm, string code)
    {
        string countryName = rm.GetString(code + ".EnglishName");
        return new Country { Code = "SE", EnglishName = countryName };
    }
}

This will help you out a lot if you ever need to make your application support multiple languages.

If this happens to be a WPF application, a resource dictionary is a much more obvious solution.

Answer 3

The decision is really: Do you want only the developer (or anyone with access to a build system) to modify the data when it needs modifying, or should an end user or possibly someone in the organisation of the end user be able to modify the data?

In the latter case, I'd suggest that a file in user-readable and user-editable format would be stored in a place where the user can access it. Obviously when you read the data you need to be careful; whatever you find cannot be trusted to be valid data. I'd probably prefer JSON to XML; I find it easier to understand and get right. You might check if there is an editor available for the data format; for example if you use a plist on MacOS X then every user who should ever go near the data will have a decent editor on his computer.

In the first case, if the data is part of your build, it doesn't really make a difference. Do what's most convenient for you. In C++, writing the data with minimal overhead is one of the few places where macros are allowed. If the work of maintaining the data is done by a third party, you can send them the source files, let them edit them, and you are then responsible for checking them and using them in your project.

Answer 4

The best example of this that exists is probably WPF... Your forms are written in XAML, which is basically XML, except that .NET is able to rehydrate it into an object representation very quickly. The XAML file is compiled into a BAML file, and squeezed into ".resources" file, which is then embedded within the assembly (the latest version of .NET reflector can show you these files).

Although there isn't any great documentation to support it, MSBuild should in-fact be able to take a XAML file, convert it to BAML, and embed it for you (it does it for forms right?).

So, my approach would be: Store your data in XAML (it's just an XML representation of an object graph), stuff that XAML into a resources file, and embed it in the assembly. At runtime grab the XAML, and use XamlServices to rehydrate it. Then either wrap it in a static member, or use Dependency Injection if you want it to be more testable, to consume it from the rest of your code.

A few links that are useful reference material:

• AL.exe (for embedding resources in assemblies)
• Resgen.exe (for converting .resx files to resources)
• XamlServices class

On a side note, you can actually use the app.config, or web.config files to load up reasonably complex objects through the settings mechanism if you want the data to be changed more easily. And you can also load XAML from the file system.

Answer 5

I guess the System.Globalization.CultureInfo.GetCulture(name) would call Windows APIs to get the data from a system file.

For loading the data in code, just like @svick had said, if you're about to load 200 objects, it won't be a problem in most cases, unless your code running on some low memory devices.

If your data never changes, my experience is simply using a list/dictionary like:

private static readonly Dictionary<string, Country> _countries = new Dictionary<string,Country>();

But problem is that you need to find a way to initialise your dictionary. I guess this is the pain point.

So the problem is changed to "How to generate your data?". But this relates to what you need.

If you want to generate data for the countries, you could use the

System.Globalization.CultureInfo.GetCultures()

get the CultrueInfo array, and you could initialise your dictionary with your needs.

And of course you could put the code in a static class, and initialise the dictionary in the static constructor like:

public static class CountryHelper
{
    private static readonly Dictionary<string, Country> _countries;
    static CountryHelper()
    {
        _countries = new Dictionary<string,Country>();
        // initialization code for your dictionary
        System.Globalization.CultureInfo[] cts = System.Globalization.CultureInfo.GetCultures(System.Globalization.CultureTypes.AllCultures);
        for(int i=0; i < cts.Length; i++)
        {
            _countries.Add(cts[i].Name, new Country(cts[i]));
        }
    }

    public static Country GetCountry(string code)
    {
        Country ct = null;
        if(this._countries.TryGet(code, out ct))
        {
            return ct;
        } else
        {
            Log.WriteDebug("Cannot find country with code '{0}' in the list.", code);
            return null;
        }
    }
}