1

I have to replace SQLAlchemy "table" (declarative) class with mostly functionally equivalent wrapper, which will use API of another microservice. All the usages can be edited, so there is no need to do complete mimicry of the SQLAlchemy features, but with reimplementing I am loosing some useful things like smart caching, connection pooling (async way), transaction support.

The "table" class is not just declaration of table fields, but also contains some higher level helper methods, an even business logic methods.

Assuming transaction support is not an issue in the case, what would be good approach(es) to deal with caching?

My approach would be to ignore caching completely, but I need to be prepared to optimize.

Solution can still use the available relational database, if it helps, with understanding that the authoritative data source for the entity in question is elsewhere. Introducing memcached or it's ilk is the second best.

It is expected a remote API call would have a bit more latency than local RDBMS call, but luckily operations are not expected to be massive, no there will be "joins" as table in question has only one place referred by foreign key (and that will be easily worked around).

1

First, I think it would be good to find a way to measure how important caching would be.

Once you've determined that you absolutely need caching, I guess a relational database would be better than using something like memcached, as you can have a very sizable cache without using too much memory. You may want to consider Redis with its persistence support enabled as well.

Make sure you treat cache invalidation with care, as you probably don't want to return obsolete information from the cache.

Optionally, though, maybe you could decouple your code a bit, by separating the business logic from the database model. Perhaps you could have your domain model class, and a helper class that would fetch it from a persistent location, such as the remote API(with or without caching enabled), or from the current SQLAlchemy-based solution. That way, in the future, you could add/replace yet another persistence mechanism without too much trouble or having to comply to a third-party interface, like SQLAlchemy's.

  • Thanks for input! As for decoupling, of course connection-level things aren't in the same class, but not sure what you mean by "database model" in this case. SQLAlchemy is exactly an attempt to model business domain and abstract from database. – Roman Susi Aug 5 '16 at 11:01
  • By "database model" I mean the SQLAlchemy-mapped class that you mentioned. You said it has some business logic in it, and I'm wondering if you want to have those methods available when you fetch objects from the remote API rather than from the database. If that's the case, it could be positive to have a separate class for business logic, and two distinct implementations for persisting/fetching it, one for the database, and one for the remote API. – Martin Jungblut Schreiner Aug 5 '16 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.