2

I'm creating a server in Java (although this question is language agnostic) that will do the following:

  1. Pull a set of "leads" from a database every 10 minutes
  2. For all leads with a certain marker, use a different database to get additional information
  3. For every lead that actually has relevant data in database #2, combine the data from two datasources and pass it to a few Machine Learning (ML) algorithms
  4. Store the results of the ML algorithms in Database #1

I'm confused as to the best way to organize it. After looking at this relevant question, I think the Repository pattern may be part of the solution but I don't think it satisfies all requirements. Particularly, I think each "lead" should be processed in its own thread, but I'd like to use a pattern that would enable me to make use of cached thread pools.

Although there may be specific Java classes or libraries that may be of use, I marked the question language agnostic as I feel the question will best be answered by suggesting pattern(s). I'll retag accordingly if the community disagrees.

1 Answer 1

1

Repository can be useful to deal with data access to the different databases.

As for the rest, depending on what you pass to the ML algorithms you might want to use a Facade that hides away the details of accessing the different database and determining if the information there is relevant. You would pass a lead to a facade class, that class would determine what markers there are and pass the lead off to different classes depending on the markers. These classes in turn add data from the other databases.

In the end, your 'base' flow would be: get a lead, 'enhance' the lead with relevant data, pass off lead to ML and store result.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.