Versioning business rules

Question

TL;DR

Storing ever-changing business rules so that an app can behave like it did at moment X in the past. Can it be done? If so, how?

Long version

At this very moment, this is more of a thought exercise than an actual need, though things could change in the not so distant future.

The company I work for is in the payroll business. We have desktop app whose job is mainly to figure out the amount of the salary an employee will receive. In Italy this is a job in and of itself, because things change literally each month, with new taxes, merges of old ones, different calculations needing to be made. In short it's a terrible, terrible mess.

Not that it makes any difference, but this is a COBOL app (don't ask) and it's quite huge, clocking in at an estimated 5M LOC. Also, saying that instead of being badly designed it has no underlying design whatsoever would be putting it mildly.

For who knows what reason, customers need to be able to recalculate past payrolls. Of course, they need the program to behave like it did in that precise month.

A little aside: personally, this makes no sense at all to me, because once you paid the guy, the only thing you might be able to do is to adjust things next month, and keeping the resulting data (which is not just the salary, but thousands of other pieces of information) in an archive for later queries would be the most trivial of exercises. Then again, I don't have anything to do with the day-to-day maintenance of this app, so the precise reasons behind these requirements might just be escaping me because of that. I might be involved at least in part with the design of a major overhaul of the whole thing, though, and that's why I'm writing this question. So, let's assume this is an actual requirement.

So what this app does each month is it copies all of the sub-apps it is composed of into a new folder, along with a complete copy of its data. A copy of almost everything, which ends up eating a tremendous amount of space on the customers' machines.

It doesn't use a database, it's still stuck with indexed files (again, don't ask).

Now the question: how would you design this if you could just start over? Whether it's turning it into a web application or making yet another desktop app, I am more interested in the conceptual side of things.

How would you design an app that needs to keep business rules versioned and reapply old ones on demand?

Now it can be done easily because they actually copy the whole shebang once per month, and every piece of data is stored in indexed files.

With a properly designed app, though, I wouldn't know how to approach this in an elegant way: database schemata tend to change, for instance, and making a copy of the whole thing each month would just be a nightmare.

But the crux of the matter is versioning business rules: can it even be done? One thing I would do for sure would be of course pulling them out of the actual code and into a set of scripts, because now, as in the best COBOL tradition, everything is hard-wired in the code, so they recompile and ship even if a simple calculation has changed.

Where to go from there, though, is a bit of a mystery.

Answer 1

The difficulty is that you have to deal with already complex rules which change every month. I would think of two techniques which can be used to mitigate some complexity:

1. Don't store historical rules in code. Unless there are a lot of accounting errors and your company needs to recalculate the payroll often, you don't need to keep all this complexity.

   At the same time, for audit, you can build an engine which builds a report of the current rules for a specific employee, i.e. a more or less programmatic, more or less humanly-readable piece of data which shows how the specific salary was determined.

   Storing those final reports will not waste a lot of space and would give enough information in a case where an employee believes he was underpaid.

2. Don't handle every possible case. It may be just too expensive to maintain the code, much more expensive than to pay an accountant who will do by hand the same work as the piece of software.

   If:

   • It takes 30 minutes for an accountant to compute the salary of an employee by hand,

   • Your company has 600 employees, 10 having a specific status which requires to apply some complicated business logic,

   • It takes two days to implement those complicated business rules,

   then you'd better to keep them out of your code and focus on simple situations. If you spend more time developing a feature than it saves to your customers, don't develop it.

---

If you really need to keep all the business rules for every past month, you may be interested by plugin architecture, with plugins being loaded on demand by the application (for example the plugin September2014 loaded when you need to create the payroll for September 2014). This will ensure you keep the application clean and keep the rules for a specific month as simple as it is possible to keep. The drawback is that you may have a lot of code duplication.

Note that in this case, using document-based NoSQL instead of ordinary SQL systems with a strict schema can help. With ordinary SQL, once you change the schema, old plugins may stop working. With NoSQL, you're free to store whatever you want in different documents corresponding to different months; you can still use legacy plugins, with the only limitation of not being able to load with old plugins documents stored with new plugins (but why would you do that?)