Multi dimensional lookups

Question

I am looking to see if there is a general design pattern or strategy to handle a use case I see often in our codebases. My best attempt to generalize this use case is "Map permutations of n parameters to a specific value, and look up the appropriate value when provided those parameters."

Due to high cardinality of some parameters we prefer not to spell out every single permutation of parameters with a massive table. In fact, we often have negation configuration (e.g. return value ABC when parameters are NOT permutation XYZ) to avoid such a massive table. Parameters can be optional during lookup. Also, sometimes we add new parameters.

For example - imagine you want to store a "support phone number" value for your store's customers. The correct phone number is influenced by their country of residence, average spend, and primary language. With 100 supported countries, 3 spend categories, and 15 supported languages, we would have to enumerate 100x3x15=4500 entries in this table to fully capture all permutations. In our real use case we see far more permutations than 4500 but this gives you an idea.

In reality, rules exist like "always use phone number 555 555 5555 if language is French", or "in Canada it is illegal to provide different services based on annual spend, so spend will not change the phone number provided". We take advantage of these rules to short-circuit lookups and avoid having to create a massive 4500 entry table.

In our code we accomplish storage and look-ups like this with disparate JSON schemas and custom code for each implementation. Is there a more general pattern/tool to handle this flow?

Answer 1

There are surely different approaches for different cases, but one approach which I did implement in the past by myself looks like this:

Use a table, but generate its content at run time by processing all rules one-by-one in advance, do not maintain the table manually.

So you can separate the rule processing and initialization from the actual lookup operation, which should simplify things a bit.

Lets take your support phone number example. Start by filling out the table with the most common cases - maybe one default phone number for each country, or maybe 3 (one for each payment level). Now, you apply the rules which define exceptions from the default initialization one-by-one. Finally, you can pass the table to a lookup function which can make use of it whenever it is required.

What you have to maintain are the rules, either in code, maybe in a parametrized fashion, or as some Domain Specific Language, so the users can edit them by themselves. Of course, you have to make sure the rules are applied in the correct order (which is also the case when using different approaches). Here, you usually have to start with the most "general" rules and then go gradually deeper into more specific ones.

The resulting table should still be so small it fits into the amount of memory you are able to afford for this process. In cases the table does have only a small percentage of filled entries, you may decide to use some data structure optimized for sparse matrixes for it.

I would not call this strategy "a pattern", just an approach with lots of degrees of freedom. When you think it is applicable to your case, feel free to leave a comment how it worked for your.