1

I have an application which needs to store different objects in different data stores based on the user's authorization credentials. I.E. the user may need to save object 1 to their BI postgres database, object 2 goes to their S3 bucket, and object 3 goes somewhere else.

Inversely, they may want to read data from different datastores based on these credentials as well. I.E records from the union of a few databases contingent on some predicate "get all rows in db1 and db2 such that the rows in db2 aren't in db3"

I'm wondering if anyone knows a good approach or design for this type of use case

  • Perhaps reading C# data adapter documentation on MSDN will give you some ideas. There are "core" adapter classes for specific databases like Oracle, et. al. Every adapter is a composite of objects including a connection object and SQL command object. It is possible to swap out objects - for example replace the SQL command object for a different query to the same DB w/ same credentials. – radarbob Apr 15 at 23:43
1

I don't know if there is such a system (I hardly doubt it, where your application is so specific to your needs pg + S3 + "Somewhere Else").

First you should create a middle layer to enforce authorization/authentication:
It may depend on the query requested and the nature of the workload you want to provide.
For example; do the users interact with your service in a request/response matter (and providing the credentials in the request) or the users interact in a session (providing all of the credentials and transact with the service over a period of time) ?

Secondly, you should strive to expose a uniform API to the user for interacting with your system through it. The user shouldn't know your implementation details and you should encapsulate it from him.
You may support only a limited functionality due to this abstraction and you'll have to translate the queries accordingly. (i.e. running such predicate as you mention where one data store is pg and the other data store is S3 may be non-trivial in some settings)

Another option is to consolidate your operation to a single data-store system (for instance limit yourself to pg) use its' expressive API / authorization facilities etc.
And only use other data stores (w/o combining them in such inter-queries) for limited purposes (for example, cold storage of logs).

(Best of luck!)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.