I'm working on a semi-complex web API, the high level design so far looks like this:
|--------| |------------| |------------|
| |--| | | |
| core | | | | |
| | | | | |
|--------| | | | |
| | controller |--| API |
| | | | |
|--------| | | | |
| |--| | | |
| engine | | | | |
| | | | | |
|--------| |------------| |------------|
From bottom to top, the core
is connected with the 'outside' (DB and third-party APIs), in the engine
is where all the processing of the data happens (I have some ML models), the data to process comes mainly from the core
, so there is a constant flow of data from core
to engine
(raw data) and from engine
back to core
(processed data). In the controller
I have all the functions used in the API (I have the cache and the prefetch system running here). And lastly, in the API module, I handle the requests and call the functions in the controller
.
I have a couple of questions:
- Is this a good architecture? I have no experience in software engineering, and I basically create this four blocks just to have the all the code organized, so I'm not sure if the architecture is well designed for scale, any tips here?
- Also, I'm not sure where should I be handling errors: This may sound like a silly question, but I'm not sure in which part of the API should I handle things like invalid parameters or unexpected errors in the code. For example, lets say I have an endpoint
/weather
with a country parameter to get the weather in this country, where should I handle an invalidcountry
parameter (or an unexpected error in the code), I initially thought that I should catch this as soon as possible and return an error message, but then the low level components wouldn't be able to handle any errors.
This is an example, (using Django for the API):
*api.py*
def get_weather(request, country):
weather = package.controller.get_weather(country)
return JsonResponse(weather)
*controller.py*
@cache # This is where the endpoints are cached
def get_weather(country):
calculated_weather = core.calculate_weather(country)
return calculated_weather
*engine/weather.py*
def calculate_weather(country):
raw_data = engine.fetch_raw_weather_data(country)
calculate_weather
*core/weather.py*
def fetch_raw_weather_data(country):
# Fetch the data using third party APIs
return data
The actual API is way more complex but I think it shows the situation. In theory an invalid country
name would raise an exception in the engine
or the core
, depending on the case, if I catch this kind of errors in the controller
, then I can return the error message to the API. I'm not sure where to catch unexpected errors, like third party APIs exceptions, or issues on processing the data, should I generate the error message there (in core
and engine
) and try to return it to the API, or should I manually raise exceptions and handle the message in the api.py
module?.
Any suggestions or tips to improve this?
API
is actually the API of this service, even if it contains logic?