I'm working on a semi-complex web API, the high level design so far looks like this:
|--------| |------------| |------------| | |--| | | | | core | | | | | | | | | | | |--------| | | | | | | controller |--| API | | | | | | |--------| | | | | | |--| | | | | engine | | | | | | | | | | | |--------| |------------| |------------|
From bottom to top, the
core is connected with the 'outside' (DB and third-party APIs), in the
engine is where all the processing of the data happens (I have some ML models), the data to process comes mainly from the
core, so there is a constant flow of data from
engine (raw data) and from
engine back to
core (processed data). In the
controller I have all the functions used in the API (I have the cache and the prefetch system running here). And lastly, in the API module, I handle the requests and call the functions in the
I have a couple of questions:
- Is this a good architecture? I have no experience in software engineering, and I basically create this four blocks just to have the all the code organized, so I'm not sure if the architecture is well designed for scale, any tips here?
- Also, I'm not sure where should I be handling errors: This may sound like a silly question, but I'm not sure in which part of the API should I handle things like invalid parameters or unexpected errors in the code. For example, lets say I have an endpoint
/weatherwith a country parameter to get the weather in this country, where should I handle an invalid
countryparameter (or an unexpected error in the code), I initially thought that I should catch this as soon as possible and return an error message, but then the low level components wouldn't be able to handle any errors.
This is an example, (using Django for the API):
*api.py* def get_weather(request, country): weather = package.controller.get_weather(country) return JsonResponse(weather) *controller.py* @cache # This is where the endpoints are cached def get_weather(country): calculated_weather = core.calculate_weather(country) return calculated_weather *engine/weather.py* def calculate_weather(country): raw_data = engine.fetch_raw_weather_data(country) calculate_weather *core/weather.py* def fetch_raw_weather_data(country): # Fetch the data using third party APIs return data
The actual API is way more complex but I think it shows the situation. In theory an invalid
country name would raise an exception in the
engine or the
core, depending on the case, if I catch this kind of errors in the
controller, then I can return the error message to the API. I'm not sure where to catch unexpected errors, like third party APIs exceptions, or issues on processing the data, should I generate the error message there (in
engine) and try to return it to the API, or should I manually raise exceptions and handle the message in the
Any suggestions or tips to improve this?