3

I'm at the point where results need to be cached to make the application more responsive.

From experience in a previous project, countless (to say the least) bugs occurred because there was lots of cached data which was never updated correctly because no one had the mental capacity to remember where everything was until someone (the end users...) noticed it and reported the bug. Now that I'm in charge this time around, I want to avoid having such mistakes happen as much as possible. I also have to deal with some very inexperienced people and would like to safeguard mistakes from happening as much as I reasonably can.

Are there any strategies or design patterns I can use to mitigate the risk of someone forgetting to update cached values?

As a hypothetical example, suppose we have a list of clients with a first and last name. Now suppose there is a suffix tree to help look up names quickly. If someone adds updates to a client's name, they may forget to update the suffix tree.

// A simple example
class Person {
    String firstName;
    String lastName;
}

At first I thought "unit tests will cover it" but with everything being very modular, such things are beyond the scope of unit tests. Then I thought about it being covered at the integration test level but then someone might not update the integration tests to have the changes, especially if they forgot about the cached values in the first place.

The next thought was to wrap everything under an encapsulating class which only exposes mutation methods, but then the setter methods might become verbose. Example:

class PersonTracker {
    SuffixTree lastNameTree;
    List<Person> people;

    void addPerson(first, last);
    void updatePerson(first, last, newFirst, newLast);
    void removePerson(first, last);
}

However the downside is the amount of code repetition, making it so only the specific class above can mutate the underlying person, exposing all the methods yet again (repeating myself in code), and combining multiple cached values into this one class where it may proceed towards some kind of monstrous 'God class' since for the issue I will be dealing with, there won't just be one set of cached values like the example we've read above.

I've thought of making each data an 'observable' class, so for example:

class Person {
    Observable<String> firstName;
    Observable<String> lastName;
}

Is this a viable solution? Or am I going down an ugly rabbit hole?

Does there exist a better way of doing this?

Each method has its own pros and cons, which one is best for a long enduring code base?

  • 1
    Possibly apocryphal, but the famous Phil Karlton quote is "There are only two hard things in Computer Science: cache invalidation and naming things." – user949300 Jan 25 at 19:53
  • 1
    im not sure your example is strictly speaking a cache – Ewan Jan 25 at 20:21
  • 5
    As long as you believe that these bugs occur because none of the developers had the proper mindset, you will not solve this problem. You should not rely on policies to keep data consistent. It sounds like you have artificially decoupled your cache from your accessors and mutators. – John Douma Jan 25 at 20:49
  • 3
    I think John Douma's comment is meant to say, so long as updating the cache is something that developers have to remember to do, developers will fail to do it. Your accessors should load the cache, your mutators should either invalidate or update it, and the rest of your code should use the accessors and mutators rather than accessing member variables directly. You can then have your accessors and mutators together, so it's easier to keep those things in sync. It may be quicker to access member variables directly, but that benefit is far outweighed by how much more complex it makes the cache. – Ed Grimm Jan 26 at 6:59
  • 1
    In other words, it should be transparent to the clients of the cache that they're using a cache. That's the power of abstractions: you don't have to learn how they work internally to use them. If you need to update the cache separately from the actual source of truth (and you have to remember to do so), you haven't abstracted it correctly. – Vincent Savard Jan 26 at 17:33
1

I am a believer that we have to embrace the stale nature of copies of data, rather than attempting to keep copies of data remain 100% up to date.

Single source of truth is a good thing; that means making fewer cached copies of data to go stale.  However, we can rarely eliminate all cached copies of data.

For example, when code reads the values in an object and presents that on the screen for a human user to read, this is a cached copy of original data being displayed on the screen as pixels in a font, not necessarily even as an object in the system.  Throw in client-server system, where the data could be stale as soon as it leaves the server.

I think are only two approaches that offer correctness guarantees:

  1. Lock every one else out of (some portion of) the system, while some user makes a change — this is actually a practical solution for most desktop applications, or,
  2. Use conditional transactions.  A conditional transaction atomically validates assumptions before executing; if assumptions fail, it aborts and notifies the caller of the failed assumptions.

When we have conditional transactions, we can read/query the model or database outside of any locking other users out, present this cached and potentially stale information to the user, and still avoid race conditions with updates.

We can improve performance, user experience, and likelihood of successful conditional transaction using dynamic refresh of cached data where the original is known to have been updated; however, dynamic refresh alone cannot guarantee freedom from race conditions the way that conditional updates can.

One approach to conditional updates is using a versioning scheme.  Versioning can be applied at varied granularity: version the whole model or database by changing a version number whenever anything in the db changes; version a row or an object in the model or database, version a field of an object, etc...

Software consuming versioned information needs to use conditional transactions for updates, and handle failures when the assumptions indicate stale information is being updated.

1

Ideally the user of your system won't have to know if data is being cached or not. You should be able to add it in, take it out, or change its behavior with no changes required in the clients.

Taking your PersonTracker class as an example, it deals with Persons. So all the add, remove, update, get and other functions should deal only with Person objects, not the various lists of parameters you may need to pass to the constructor. So addPerson should take a Person (whether by smart pointer, raw pointer, some kind of reference, or value is a separate discussion). updatePerson would take two: the original record, and the updated (modified) record. Get methods should probably return a copy of the Person, to make management of the internal structures easier and less error prone, although your exact use case may limit this option and require the additional complexity of references to your cached data.

You could also provide helpers to simplify things, like updatePersonLastName that would take a Person to update and the new value for the last name.

The suffix tree updates would happen within these methods, without any additional effort by the caller.

Then there's the issue of error handling: How to handle adding a person that is already there? Updating one that isn't there because it was already updated or removed?

  • I'd suggest adding a mention of the possibility that the caching behavior may be required to change in the future: centralizing cache handling to the i/o route makes it easier to ensure that the behavior actually gets updated. – aerohammer Jan 27 at 3:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.