0

So I have this issue. My website uses data, that is scraped from a different site - sports results. This data can update in relatively random intervals, but I do not care if my data is a bit stale - it does not have to be instant, but it should update on some regular basis.

At the same time, I cannot just cache the responses from the external site -> I process them and import into a graph database so that I can do other analytics over them.

I would like to have a system like this:

interface IDataSource {
 public function getData(): array;
}

class ExternalDataSource implements IDataSource {
// gets data from the external website - the ultimate source of truth
}

class InternalDataSource implements IDataSource {
// gets data from my own graph database
}

class InternalImportDecorator implements IDataSource {
  private $external;
  public function __contruct(ExternalDataSource $external) {
    $this->external = $external
  }

  public function getData(): array
  {
    $data = $this-external->getData();
    // import the data into my internal DB
    return $data;
  }
}

class CompositeDataSource implements IDataSource
{
    public function __construct(ExternalDataSource $external, InternalDataSource $internal)
    {
        $this->external = new InternalImportDecorator($external);
        $this->internal = $internal;
    }

    public function getData(): array //HERE I NEED HELP
    {
      if(rand(0, 100) > 95) {//in 95% of the cases, go for internal DB for data - like weighted load-balancer somewhat
        //here I need something like "chain of responsibility" in case the internal DB is not yet populated
      } else { // go for the external data source, so that I can update my internal data
        //what if the external data source is not available? I need a circuit breaker with fallback to internal
        //what if I fall back to internal and the internal DB has not yet been populated
      }
    }

}

I have a general idea about the code and the composition, I just need help with one method implementation. Or maybe just some nomenclature, how is this situation properly called, so that I can google it myself.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.