Keep Hitting 3rd party API or Store Commonly Queried Data In DB?

Question

Colleague and I have begun a long (and hopefully fruitful) project. I've been building and have come to my first crux, and now I ask the experts of the Stack Exchange for their guidance.

I'll keep this simple. Our front end has functionality to search for stocks and get back real-time intraday data on it. It's a search dashboard for info, nothing else YET. Our front end sends the user's search request to our StockDataAPI, which actually just sends a GET req to some public trusted stock data api.. the user doesn't know the difference, they think we own the publicly available data.

Asking: Is it right to keep this approach? Over time, wouldn't this become clunky and burdensome? Would it be better to have the following: Save the returned stock information in a database for easier lookup? But checking the entire DB before querying 3rd party API will become cumbersome once DB is very full. I'd also have to have a step to save info in DB if the information being returned wasn't present in the DB. Or should I try something completely different?

Thanks in advance for all your time!

Answer 1

I would have thought you would want some form of caching to avoid sending multiple requests to the third party within the same second.

This needn't be a database, although I see no reason why a properly indexed db would become clunky as it filled up.

But aside from the cache, there could be some business value in keeping the old data. It could potentially allow you to add features not available through the third party.

I can imagine that allowing customers to see a history of stock prices for example would be a nice feature