Is there a common pattern for handling missing data found deep in an application?

Question

I have a manufacturing management application that I'm in the middle of redesigning to make it more universal (we acquired a small company and they will use the tool as well). Where before I had many values hard coded because the application had a single client, now it's more configuration based with various data elements coming from options files or database tables. This is rewarding in that it's a better design and the application is evolving, but it has presented a new challenge: low level services and components encountering situations where they are missing data.

For example I may have a serial number generation component that used to have a hard coded string it would use as a part of the serial number, now that "base serial number" is dynamically generated and coming from a format string stored in the product table. There are several layers and classes between the product table data and the component generating the serial number. I want to handle the missing data exception and allow the user to correct the problem in the current application session rather than blow up and crash.

I see two possible approaches:

1. Add deep data validation at the point of request to make sure everything is there to avoid encountering exceptions at the low level code. I don't like this because it means my presentation layer would need to either have knowledge about the objects and data involved downstream or I would need to sprinkle all my objects with "HasRequiredData()" methods.

2. Create a new Exception type that represents the general problem of missing data. When a low level component encounters a situation where data it needs isn't available it would throw this exception. Presentation layer then handles this exception nicely.

So my question is: Does option #2 seem like a good approach and design to you all? Is there another common approach to issues like this?

Answer 1

The common handling (as I see it) is to raise an exception and log the error. You always will have cases where business logic comes to places where nobody has been before. That means you need to handle the exception gracefully:

• inform the user about the exception,
• clean the stack,
• log the error and
• later on check the log regularly for messages to clean up and provide fixes.

Answer 2

If your process is long running or mission critical, prevalidation is a requirement.

Deep validation doesn't require breaking separation of concerns. One solution would be to make every component consuming configuration to expose prevalidation method (as components are nested, these methods could be nested too). Then, on a stage of configuration change, these methods would be able to provide a comprehensible configuration defect description to the user performing reconfiguration.

Such precondition check is often used with a command pattern - a composite command executes precondition checks for all children before it allows execution of self.

Answer 3

When your program stumbles over some missing data it needs to work correctly, which cannot reliably be replaced by some meaningful default value, it should throw an exception. Otherwise, this would mask severe errors, lead to wrong or unexpected results, or just crash the application, which is probably not what you want.

However, be careful with the proposed solution of "handle the missing data exception and allow the user to correct the problem in the current application session". If your program starts asking the user for entering missing data at the point in time when the program notices the problem, it may be in the middle of a transaction, or it will simply not fit into the working process of the user. For example, the user who is dealing with the current use case might not have the missing values available at that point in time, or he might not be the person who can prepare the data, or the data needs some special quality check by a second person, and so on.

So - depending on the actual use case - having an additional "deep data validation" step before the main use case starts might be the correct thing to do, with the option of solving the problem before the use case starts. This should be independently from the fact your program throws proper exceptions.

You won't find a "common pattern" how to deal with this, because this is not a "one size fits all" situation. Whenever you introduce configuration data into a business process where formerly hard coded values have been used, you need to analyse thoroughly who will be responsible for managing this data. Formerly, it was the developer of the application. Now, it can become someone else. Some of this kind of data might be provided best by an administrator, some by the user itself, some by special "power" users, maybe in a use case preparation step. Sometimes, the developers can provide good default values for this data, but give the users the availability to change them later. And some of this data may still best be managed by the developers exclusively.

You will have to decide this for each and every single attribute of the data in stake, there is no easy way around this.