Uncertain of cache solutions for CAD models

Question

I am running a personal project of offering 3D model generation as a service (in this case CAD model). Since most of the generation of 3D models take a long time, I thought about using cache to improve performance for the users. The 3D models are generated based on well-known parameters, and those would be used to access the data from the cache.

Generated CAD models are saved on a file system. The service would return those files as a byte array. It is hard do specify the frequency in which these models will be generated, since the service is not online yet.

I have been searching for a cache solution for a project of mine that has the following spec:

1. Database storage of cached data. In memory caching would not be good here since CAD models can go all the way up to 40 MB each. Also, I would need to distribute the cached data acess accross multiple servers, with all of them caching data to the same storage.
2. Upfront + lazy population, along with monitoring of hits on the cache.
3. Explicit eviction via command. I want every CAD model to be cached, but, I also want to be able to explicitly kill the entries that match a certain key (via request, for example).

I have taken a look into some solutions:

1. Redis: As I mentioned earlier, I think it is not a good idea to store this type of data in-memory, since the models will be cached without a time/access based eviction policy (unless this isnt really a problem... correct me if I am wrong).
2. CouchBase and MongoDB (well-known NoSQL database solutions): Those would provide a database to store the models, sometimes with a query cache layer (in case of CouchBase). Although, it seems to me like this is more of selecting a database, instead of selecting a cache solution. Even if I go for a NoSQL database for caching, I will still have to develop most of the logic behind eviction and cache access/storage.

I have been looking for a cache solution that fits my spec with standard functionalities. I am trying to avoid developing any of that myself, but it seems to me right now that using a NoSQL database with some development to store/access data is my best bet.

Can anyone more experienced with caching give some insights on that matter? Am I going right about it?

Answer 1

I think implementing this using something like SqLite looks pretty straightforward, with

• just one table containing the parameters (as primary key) and
• one BLOB column for the CAD model, and
• a few columns for statistics.

Yes, you will have to implement the logic to manage the cache by yourself, but the particular population and eviction requirements you mentioned don't look very complex to me. I think it is unlikely you will find an "out-of-the-box" solution easily which suits your needs. If you expect the structure of the parameters to change often, a No-SQL solution may be suited better, of course.

I doubt SQL or NoSQL willl make a big difference for what you are trying, but I would actually keep things as simple and lightweight as possible at the start. You can always switch to something more complex when it really turns out the simple solution is not sufficient. MongoDB and CouchDB look both larger and more complex than you may need (but I am not an expert on those systems).