I have a requirement where I need to validate excel file before importing into application. Excel file will have around 15-20 defined columns. And we are having around 200-250 rules to validate the excel file.
Right now, one windows service is there, which is scanning the directories to process this excel files. windows service can process multiple files using multi threads. But single file will be validated row by row only. Currently validations has been written in single class using methods.
It is working fine so far but now we are getting requirement changes in rules and validations and I want to make rules and validation configurable and more manageable.
Examples of Rules:
- There should not be a Empty Values.
- Check Max and Min Length
- Date formats
- If value present in one column, other related columns value should be empty.
My questions are:
- How to Make Rules and Validation Configurable? I've checked Enterprise Library Validation Block. Which could be used using XML configuration. But I feel it will not be maintainable going ahead.
- How to make Rules and Validation Turn On and Off on the fly without changing and code and compilation? I want to achieve design somewhat like SonarQube Rules and Validation.
- Right now, using multi threading multiple files are being processed. Is it Ok to process single file records parallel as well? I don't think it will increase the performance as all thread being used for processing multiple files.
- Any design pattern for such kind of problem? I think Rules Pattern along with Pipeline kind of design could be used.