2

I am working currently on a project that requires a series (almost 86) calculations to run based on a user data input. The problem is that each calculation has a series of requirements:

  • Should be able to hold a version variable to be able to distinguish changes on each calculation algorithm implementation. This way, anytime we modify an algorithm, we know which version was used on specific calculation.
  • Should be able to load specific data from other modules within the application (namely, we have 8 entities) so each one can choose the information necessary for its operation.
  • Should be able to determine if it's "runnable", and by which we would write a function (?) that verifies that the data extracted (from the previous requirement) meets some custom criteria for each calculation that guarantees the algorithm will be executed correctly.
  • Should have each a different algorithm implementation.
  • Generate a series of execution metrics (logs) and store them, such as data fetching time, algorithm execution time and the sampleSize which refers to as the amount of data loaded for each specific calculation to run.

Currently, what I've done is: created an abstract class Calculation with this structure:

abstract class Calculation<T, F> {
  /**
 * Logging Variables.
   */
  private initialDataFetchTime: Date;
  private finalDataFetchTime: Date;
  private initialAlgorithmTime: Date;
  private finalAlgorithmTime: Date;

  // Final result holding variable.
  private finalResult: T;

  // The coverage status for this calculation.
  private coverage: boolean;

  // Data to use within the algorithm.
  private data: F;

  // The version of the Calculation.
  public abstract version: string;

  // The form data from the User to be used.
  public static formData: FormData;

  /**
 * This is the abstract function to be implemented with
 * the operation to be performed with the data. Always
 * called after `loadData()`.
   */
  public abstract async algorithm(): Promise<T>;

  /**
 * This function should implement the data fetching
 * for this particular calculation. This function is always
 * called before `calculation()`.
   */
  public abstract async fetchData(): Promise<F>;

  /**
 * This is the abstract function that checks
 * if enough information is met to perform the
 * calculation. This function is called always
 * after `loadData()`.
   */
  public abstract async coverageValidation(): Promise<boolean>;

  /**
 * This is the public member function that is called
 * to perform the data-fetching operations of the
 * calculation. This is the first function to call.
   */
  public async loadData(): Promise<void> {
    // Get the initial time.
    this.initialDataFetchTime = new Date();

    /**
     * Here we run the data-fetching implementation for
     * this particular calculation.
     */
    this.data = await this.fetchData();

    // Store the final time.
    this.finalDataFetchTime = new Date();
  }

  /**
 * This is the public member function that is called
 * to perform the calculation on this field. This is
 * the last function to be called.
   */
  public async calculation(): Promise<T> {
    // Get the initial time.
    this.initialAlgorithmTime = new Date();

    /**
     * Here we run the algorithmic implementation for
     * this particular calculation.
     */
    this.finalResult = await this.algorithm();

    // Store the final time.
    this.finalAlgorithmTime = new Date();

    // Return the result.
    return this.finalResult;
  }

  /**
 * This is the public member function that is called
 * to perform the coverage-checking of this calculation.
 * This function should be called after the `loadData()`
 * and before `calculation()`.
   */
  public async coverageCheck(): Promise<boolean> {
    // Execute the check function.
    this.coverage = await this.coverageValidation();

    // Return result.
    return this.coverage;
  }

  /**
 * Set FormData statically to be used across calculations.¡
   */
  public static setFormData(formData: FormData): FormData {
    // Store report.
    this.formData = formData;

    // Return report.
    return this.formData;
  }

  /**
 * Get the coverage of this calculation.
   */
  public getCoverage(): boolean {
    return this.coverage;
  }

  /**
 * Get the data for this calculation.
   */
  public getData(): F {
    return this.data;
  }

  /**
 * Get the result for this calculation.
   */
  public getResult(): T {
    return this.finalResult;
  }

  /**
   * Function to get the class name.
   */
  private getClassName(): string {
    return this.constructor.name;
  }

  /**
   * Function to get the version for this calculation.
   */
  private getVersion(): string { return this.version; }

  /**
   * Get all the Valuation Logs for this Calculation.
   */
  public async getValuationLogs(): Promise<CreateValuationLogDTO[]> {
    // The array of results.
    const valuationLogs: CreateValuationLogDTO[] = [];

    // Log the time the algorithm took to execute.
    valuationLogs.push({
      report: Calculation.formData,
      calculation: this.getClassName(),
      metric: 'Algorithm Execution Time',
      version: this.getVersion(),
      value:
        this.initialAlgorithmTime.getTime() - this.finalAlgorithmTime.getTime(),
    });

    // Log the time to fetch information.
    valuationLogs.push({
      report: Calculation.formData,
      calculation: this.getClassName(),
      metric: 'Data Fetch Load Time',
      version: this.getVersion(),
      value:
        this.initialDataFetchTime.getTime() - this.finalDataFetchTime.getTime(),
    });

    // Sample size is calculated and not an issue for this matter.

    // Return the metrics.
    return valuationLogs;
  }
}

And then, created subsequent classes for each calculation that extend the previous class, such as:

export class GeneralArea extends Calculation<number, GeneralAreaData> {
  /**
   * Versioning information.
   * These variable hold the information about the progress done to this
   * calculation algorithm. The `version`  field is a SemVer field which
   * stores the version of the current algorithm implementation.
   *
   * IF YOU MAKE ANY MODIFICATION TO THIS CALCULATION, PLEASE UPDATE THE
   * VERSION ACCORDINGLY.
   */
  public version = '1.0.0';

  // Dependencies.
  constructor(private readonly dataSource: DataSource) {
    super();
  }

  // 1) Fetch Information
  public async fetchData(): Promise<GeneralAreaData> {
    // Query the DB.
    const dataPoints = this.dataSource.getInformation(/**  **/);

    // Return the data object.
    return {
      mortgages: dataPoints,
    };
  }

  // 2) Validate Coverage.
  public async coverageValidation(): Promise<boolean> {
    // Load data.
    const data: GeneralAreaData = this.getData();

    // Validate to be more than 5 results.
    if (data.mortgages.length < 5) {
      return false;
    }

    // Everything correct.
    return true;
  }

  // 3) Algorithm
  public async algorithm(): Promise<number> {
    // Load data.
    const data: GeneralAreaData = this.getData();

    // Perform operation.
    const result: number = await Math.min.apply(
      Math,
      data.mortgages.map(mortgage => mortgage.price),
    );

    // Return the result.
    return result;
  }
}

/**
 * Interface that holds the structure of the data
 * used for this implementation.
 */
export interface GeneralAreaData {
  // Mortages based on some criteria.
  mortages: SomeDataEntity;
}

The idea is to allow us to perform three basic operations:

  1. Load the data for every calculation.
  2. Validate coverage for every calculation.
  3. If the previous step returns a general "true", run calculations.

However, this pattern has raised some issues as the FormData (the information the user uploads) is stored statically, which means that if some calculation is already running and another user performs an upload, I cannot set the FormData because it will cause the other User's calculations to go nuts. However passing the FormData to each function constructor seems like a lot of work (if you feel like this should be the way, I have no fear of writing code ;) )

Perhaps is this quarantine, however, am I not seeing something here? Currently, the final execution looks something like this:


public performCalculation(formData: FormData): Promise<FormDataWithCalculations> {
  // Set general form data.
  Calculation.setFormData(formData); // <--- Error in subsequent requests :(

  // Instance Calculations.
  const generalAreaCalculation: GeneralAreaCalculation = new GeneralAreaCalculation(/** data service **/);
  // 85 more instantiations...

  // Load data for Calculations.
  try {
    await Promise.all([
      generalAreaCalculation.loadData(),
      // 85 more invocations...
    ]);
  } catch(dataLoadError) { /** error handling **/ }

  // Check for coverage.
  const coverages: boolean[] = await Promise.all([
    generalAreaCalculation.coverageCheck(),
    // 85 more coverage checks...
  ]);

  // Reduce coverage.
  const covered: boolean = coverages.reduce((previousValue, coverage) => coverage && previousValue, true);

  // Check coverage.
  if (!covered) { /** Throw exception **/ }

  // Perform calculations!
  const result: FormDataWithCalculations = new FormDataWithCalculations(formData);

  try {
    result.generalAreaValue = generalAreaCalculation.calculation();
    // 85 more of this.
  } catch (algorithmsError) { /** error handling ***/ }

  /*
   (( Here should go the log collecting and storing, for each of the 85 calculations ))
  */

  // Return processed information.
  return result;
}

I am not afraid of writing too much code if it means for it to be reusable, maintainable, and more importantly, able to be testable (oh yes, test each calculation to ensure it does what it is supposed to do in normal and edge cases, that's why classes were my approach so each one would have attached a test), however I am completely overwhelmed with writing this tremendous amount of code instead of just writing 85 functions (which is what is already been used) and call each one of them.

Is there any patterns? Guidance? Advice? Reference? Study material? I can't seem to reduce this code any more but wanted to ask in case someone knows a better pattern for this kind of problems and, in case it is useful, the code in TypeScript (NodeJS with NestJS API) to understand how is everything getting wired.

Thanks in advance and apologies for my awful english!

2
  • 2
    One piece of advice, if calculations reach the point of struggling to understand "how they're wired", is to try and stay consistent to how the calculations are performed on paper. Computers can perform calculations faster and more reliably than people, but they do nothing to reduce the intrinsic complexity, and if the whole thing cannot be described and worked on paper, then you've probably moved past the point where anyone is truly capable of understanding the calculation.
    – Steve
    Mar 31, 2020 at 8:45
  • @Steve that's exactly what we're trying to solve :) most of our calculations need to be revisited (adjusted would be a better word for it) based on some internal parameters, it's done 2-3 times a week and people that will implement calculation logic is different from the main API team. I am just trying to give a flexible structure for them and us to mantain those conditions and not worry about extra-complicated, separated, data-fetching, coverage validation and calculation execution! Totally agree that probably I need to look at calculations to get a better perspective :) Thank you very much! Mar 31, 2020 at 18:51

1 Answer 1

2

The most simple and safe solution I can up with is to make FormData an immutable class (so when the form data is shared and passed around between different calculations without copying, there will no side effects, even in a multithreaded context). Then pass one FormData object into the calculation where it makes most sense and reuse this "freezed" set of data throughout the calculation.

This might involve some work in the code where the FormData gets uploaded to make it immutable. But if that ends up in "a lot of work" in the calculation code, I think you are doing something utterly wrong: holding the data in a new member variable and using the data for the individual calcucation from there instead from a single, shared static global variable looks to me only like a slight change, not a huge one.

2
  • Totally making FormData immutable would solve the problem! The thing was that I've never had to make a calculation with this amount of requirements and (honestly) were expecting someone that would tell me this idea, in general, was insane! Thank you very much, making the FormData immutable and passing it to each calculation is definitely the way to go. Do you think everything else makes sense? :) Thank you very much! Mar 31, 2020 at 18:58
  • @HumbertoWoody: at least, it does not look unreasonable to me. Solving problems by decomposing a data flow into several steps, with uniform interfaces and uniform concepts for controlling and validating the steps is a praxis-proven programming model which I have applied by myself sucessfully in the past (to some degree), and which I have seen working well a few times in larger contexts. If all those features mentioned in your question are really required is something you have decide by yourself, that is nothing an outsider can tell you.
    – Doc Brown
    Mar 31, 2020 at 19:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.