0

Context - I'm building a flight booking system (online travel agent) that will partner many airlines to sell their seats.

I've designed my architecture to work like this:

  1. When a user searches for a flight route, a request goes to look up a cache
  2. If the cache has available flight inventory, return it to the user
  3. If it does not, create a job in a queue
  4. Sometime later, a queue consumer will pick up the job and hit the airline's API / website to retrieve flight inventory. Once retrieved, they will be placed into the cache.
  5. The code in #1 polls again, retrieve the inventories in the cache, and display the search results to the user.

Benefits of this setup so far:

  • the queue / asynchronous fetching approach provides a buffer to prevent hitting airline systems too much
  • the cache provides very fast response

Problems encountered:

  • Very high memory usage on the cache due to the huge variety of flight routes available
  • For routes that are not searched often, users will need to wait a while as the system is polling to get the search content
  • If an airline gone down, then there won't be any inventory data available, causing the user to see nothing + loss of sales

How will you improve the design?

———— EDIT ————

More context:

  • In the travel industry, every airline has a Look-To-Book ratio given to distribution partners. For example if you hit their API / website too often but not book, then you will be penalized.

  • In addition, every airline also stipulates maximum number of clients able to connect from a partner.

  • This distribution site works with hundreds of airlines and serve millions of requests a day.

  • Latency to the airlines easily take from 2s - 15s. They are not too tech savvy.

  • There are additional middlemen partners too like GDS (example: Amadeus) to pull flight routes from.

1
  • 2
    Just FYI, there are companies with hundreds of employees specialising in providing this kind of thing as a service, because once you get beyond short-haul "bucket and spade" routes, things get really complex really fast.
    – IMSoP
    Apr 16 '20 at 16:27
1

On your design concerns:

  • You can't book anything if the airline system is down, so lost sales for you and the airline are inevitable.

  • On the cache size: do your design calculations for the amount of memory involved, and ask whether it is really a problem... If you have your cache on a separate machine you can throw hardware at the problem and equip it with 128GB of memory, plus a few 2TB SSDs... That should hold quite a few days of airline routes. Airline changes to routes will limit the useful data you can hold: you probably can't use routing information that is more than a few hours' old without confusing customers by offering routes that have already been changed by the airline. (Obviously you also have to validate the cached information when booking).

  • So then the question becomes what are the likely delays when hitting the airline API for rarely used routes. You can't do anything about these. You can't (unless the airline lets you) pull down its entire database. But you can do some design calculations to see if these delays matter...

One criticism of this design is the use of polling. If there is a delay due to an overload, all the clients will be polling for cache results, causing in an even more loaded system. It has the benefit of simplicity, but the drawback of potential catastrophic behaviour when systems have delays.

Lot's of ways to tweak your design, but not enough information to know if it is appropriate. Do you need load balancers? Multiple servers? Multiple workers querying airline APIs? Redundancy? Somewhere to store bookings (database)? Backup systems? Auto-scaling? Location of servers? Content delivery network for edge caching? What's the engineering budget? What are the engineering trade-offs you want to make if that budget isn't enough to do everything? Does this have to fit onto a particular set of cloud services? Etc. etc.

=== Edited to add ====

The extra business information makes your caching design much more understandable.

The long API delays might suggest using pre-caching for common API queries which are likely to be made during the following time period. Obviously there's a trade-off here between this and the look-to-book ratio.

At the UI level you need to "entertain" the user while the queries are being made. If you look at any of the travel websites (e.g. expedia) you'll notice a screen or animation showing something like "searching for the best possible route..." or "reticulating splines..." so the user expects a long delay at this point but perceives it as adding extra value.

To improve the architecture, consider some of the failure modes: what if an airline API stops working? (Retry? Second worker retries job?) How many workers per API end point? Web Host fails? Cache fails? Database fails? Network fails? What redundancy makes sense? What monitoring makes sense? How does an operator know if something is wrong (internal or external)? e.g. DNS server fails for airline API? Airline API stops responding? What actions can an operator take? What happens if a machine fails and has to be rebooted? Look at every single process and network communication in the architecture and ask what happens when it fails, and how it can be determined if it has failed.

I'm assuming this is a class exercise... so some of these risk scenarios might be easier to ignore (accept the outage) than if you were set on re-writing Expedia...

3
  • I provided more business context to justify my design choices.
    – unclelim12
    Apr 19 '20 at 4:56
  • Hmmm if polling is not a good solution, what about a reactive approach whereby clients get notified whether there’s a change in inventory? How would you design this?
    – unclelim12
    Apr 19 '20 at 5:00
  • @unclelim12 a solution where the client gets notified when the cache entr(ies) becomes available - but I'm not sure whether by client you mean the middleware or the end user here.
    – MZB
    Apr 20 '20 at 17:16
0

So basically a cache is a nice pattern to reduce the number of IO calls if requests are unevenly distributed. I'm not sure whether this is the case in the tourism business since people tend to buy tickets far ahead. So you can put in a cache just a small amount of hottest destinations with "last frequent out" policy of cache invalidation.

This will handle the following concern

Very high memory usage on the cache due to the huge variety of flight routes available

Alternatively, you can come up with a huge cache i.e. in Redis cluster but I think it's worthless because I assume that there is almost event distribution of user requests.

If it does not, create a job in a queue

Let me be clear. Do you really assume that the load on your service will be that big so you can DDoS airline servers with requests? Furthermore, if you put the most frequent requests into cache this makes me wonder whether you anticipate that you will be able to DDoS airline servers with unique requests. I think it's a highly optimistic prediction. That's why my thinking is that the queue just adds an additional burden.

Furthermore, I think if you operate with a lot of airlines then you should expect your servers to be a bottleneck, not airline servers. So you have to design your services so that they will be horizontally scalable. It worth noting that the cache should be distributed, not in-memory. So that multiple instances of your service would work with the same cache.

If an airline gone down, then there won't be any inventory data available, causing the user to see nothing + loss of sales

But I can't see any other options here. I mean one wouldn't expect to be sold a ticket that cannot be confirmed by an airline due to some technical reason, arrive at the airport only to know that the ticket is invalid. The only thing you can really come up with is some sort of wise retry policy if this is a temporary outage.

Also, I think that you need a background job to check whether your cache is up to date. Flights may be cancelled due to some unforeseen circumstances. I.e. outbreak of global pandemia.

2
  • 2
    The reason for queuing jobs might be to handle airline API rate limiting rather than concern for overloading their servers.
    – MZB
    Apr 18 '20 at 4:28
  • That’s right MZB, thanks. Updated my question with this context
    – unclelim12
    Apr 19 '20 at 5:03
0

Cache is a good option to avoid load that’s hiding the actual servers (airline services) in your case. I presume you are not opting for in-memory cache within the web server instead individual caching servers like Redis or Memcached. I would suggest Redis since supports warm start during reboots and also supports cluster support.

But while caching we have to be careful of the information stored,

  • How much size it is?
  • How long it is going to occupy the cache (stick with some timeout)?
  • What is the eviction policy you are associating with your cache?

Caching alone cannot improve your latency because your cache would be hit by your web service (hosted at some region) – so we need to think about whether do we need regional based data center (or) region-based web server to support the customer.

Instead of queue I would suggest to have a front ending web server which hits the cache returns the value to the customer. If there no data in cache, contact the airline (may be an api call here) – fill the cache and return to the customer. The disadvantage with the queue what I see here is that it may not prioritize the current user requesting for information.

Based on the data available in cache you can use it for showing the information. For booking you can validate the data available in cache with flight booking service and book the tickets. If a particular api/ particular domain or particular flight booking service is not available you can queue the request here by showing message to the customer that there is a temporary service unavailability we would retry after sometime and complete the transaction and repay the customer when the retry is not successful.

For retry interval you can consider options like exponential back off.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.