How to design sequential requests to third-party APIs?

Question

I need to implement the following flow:

Back-end makes HTTP request to Service A, waits for a response from Service A,
then makes HTTP request to Service B, waits for a response,
then makes concurrent HTTP requests to Service C, waits for responses.

if the response from Service A does not contain specific info required for the next request, break the flow and return an empty response.
the same is true for the response from Service B.

Services A, B, and C are different external APIs.

Question: Could you suggest a design pattern for this flow? My goal is to make the project maintainable. In case if tomorrow we'll need to replace Service C with Service D, or add new Service E to the flow.

I'm sure failures are inevitable... So my second question is how to handle network errors which may occur during any of the mentioned requests? Should it be a single try-catch block wrapping the flow, or separate try-catch blocks for each request?

I'm using Java/Kotlin with Spring framework if this is important.

---

I'll handle response caching, authentication, token refresh, logging, and configuration in separate classes. This question is specifically about the flow design.

Answer 1

When talking about a sequence of comparatively expensive tasks, you want to look at the workflow and pipeline patterns. What you will get is a series of independent, individually designed and tested modules that can be combined or composed into a workflow. It will provide component reuse and sometimes is configured externally or dynamically.

You will also gain quite a bit of complexity because someone will provide the threading and the continuation logic But at the end of the day, these services are called in sequence, functionally it will be the same.

My experience is with the .net parallel task library, I think the code example will illustrate the concept well (quoted from MSDN). Here we have two inline functions, one to download, one to process. The runtime by default will run one thread each. It will provide buffering of tasks. The HTTP call is asynchronous, which means the thread is not blocked waiting for the HTTP response.

  var downloadString = new TransformBlock<string, string>(async uri =>
  {
     return await new HttpClient().GetStringAsync(uri);
  });

  // Separates the specified text into an array of words.
  var createWordList = new TransformBlock<string, string[]>(text =>
  {
     // Remove common punctuation by replacing all non-letter characters 
     // with a space character.
     char[] tokens = text.Select(c => char.IsLetter(c) ? c : ' ').ToArray();
     text = new string(tokens);

     // Separate the text into an array of words.
     return text.Split(new[] { ' ' }, StringSplitOptions.RemoveEmptyEntries);
  });

  downloadString.LinkTo(createWordList, linkOptions);

I found a thing called activiti that might be useful? Again, this might be overkill, depending on how often your workflow actually changes, and you could just work with promises for a cleaner implementation.

Answer 2

You can wrap each Service into it's own class, where all the classes implement a common interface. This would allow you to potentially change the service in the future without the client code (i.e the code that organises the flow) knowing which service is being called. The client code would only need to know that these 3 services need to be called in a particular sequence.

Network errors are potentially already handled by Java and would simply result in the request failing altogether.