There are different types of financial products and I am trying to simulate them in different scenarios. Each of the product types requires different input data. Suppose the input data are organized with a database table, then the input data for a certain product may have its own total number of columns, and the columns may also have different business meaning compared with other types. In addition, for each product type, there can be a large number of actual product for the type (such as 500,000), and each product would be a data record in the table.
For example, input data for product type A may include fields "a_1", "a_2", "a_3"..."a_20". Input data for product B may include "b_1", "b_2", "b_3"..."b_25". It's not possible to clearly define and exhaust the input data structure for all product types in advance. A user may introduce new product types with new input data requirements.
While the input data vary in terms of the number of data elements and their business meanings, the data elements across product types can be abstracted into a few groups and each group can be handled in the same way, using common calc engine implemented using c#.
Users prepares an input data file for a product type. Users then separately use GUIs to tell the calc engine what groups of data are included in the input file. The calc engine uses the info from GUI to interprets the data from input the file and then perform the calculation.
I am trying to pass the dynamic, varying input data to the calc engine.
Have a one-size-fits-all data structure for all product types. This will require a large large number of data fields to hopefully meet the needs of the most complex data type. In addition, I am also trying to use objects to represent the data records. With this approach, the object would also include unused data fields. Considering the large number of data records (and therefore objects)for a product type, this approach can be challenging.
This approach will also require generic field names, such as "field_1" and "field_2", instead of "a_1" and "a_2". The generic field names can be confusing and additional mapping to actual user field name may be necessary.
I was thinking maybe I can have a range of generic data structures, like "a-few-sizes-fit-all" instead of one-size-fits-all. Maybe I can have a data structure 1 to meet the needs of up to 20 data elements, a data structure 2 to meet the needs of up to 30 data elements, and so on. This may help clean the data table and reduce the size of objects. In this case, the calc engine will need to deal with different data classes, with the classes are still known at compile time.
Dynamically generate classes for the different product type, and have the calc engine deal with the classes generated at run time. This may have performance concerns. It also makes the calc engine more complex by asking it to deal with unknown data classes.
I wonder, if my thoughts (with the attempted solutions) about the input data structure and the use of objects to represent data records are reasonable. Is there a better solution? Thanks!