Got an organization that its core work is acquiring various sources of data, validating it and integrating it to existing mostly financial data sets and then availing that data to many different types of customers through various web, mobile, USSD and API applications it has developed internally and others which have been outsourced.

Some of the challenges they are having and will continue to have is enforcing application authentication to access the database, managing DB sessions and importantly, ensuring that the various developers from different countries and organizations return the same result as other applications.

We want to enhance security by encrypting the DB -Postgresql DB- and want to be able to manage access to it. There is also more development work for new applications but all new applications will now be fully outsourced.

Since data is the core asset of this company, is it advisable that the company provides one API that all other applications call when they want data from the company? Will this be a bottleneck to manage and maintain or its actually the best way to go about in such an environment? By such an environment I mean the core job is data aggregation, securing the data and then giving access to business partners who are building web apps, USSD applications, SMS applications etc making hundreds of thousands of calls a day.

closed as too broad by Robert Harvey, user40980, Scant Roger, GlenH7, Matthieu M. Dec 21 '15 at 12:40

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 2
    I don't understand your question. Of course it's been done before. Your choice of endpoints will depend on your system requirements. Generally, having a single API simplifies things, but a single API might not be the best approach for your specific system. Again, it depends on the requirements. – Robert Harvey Dec 17 '15 at 16:05
  • Apologies for the "has it been done before" quip if it makes me seem naive :-) Either way, I tried to explain the ecosystem. Tons of data, multiple applications and end points, external developers, database encryption, high level of security and control required. After that each application has its own requirements and its own clients. But for the Company itself, managing and securing and giving access to that data is its core requirement – lukik Dec 17 '15 at 16:17
  • Your tradeoff will be the effort it takes to secure all of those endpoints versus the difficulty in marrying them to a single endpoint to rule them all. The best approach is probably going to be somewhere in the middle: reduce the number of endpoints as much as possible by combining common functionalities. – Robert Harvey Dec 17 '15 at 16:20
  • 3
    It is a great idea. Making an explicit api only use to provide the layer between the database and clients usually leads to completely moving all the DB logic into this API, which is very good, because it removes the logic from the database itself. It also makes your API more scalable, if it is well written. The API can work with master/slave databases, and also be copied among more servers to which the requests are redirected using a balancer. – Andy Dec 17 '15 at 16:24

If you do heavily custom things like your own layer of authentication and encryption, it's a good idea to have a canonical, well-supported API. It's easier to maintain one API than several.

I'd try to have a single low-level API, something like a layer that can pass through clients' SQL while handling auth and encryption transparently. Higher-level interfaces can be built on top of it, by your team and other teams alike.

With some luck and consideration, the low-level interface can be built in such a way that standard tools will access it as is. E.g. providing a JDBC and/or Python DBI interface on top of your custom layer would greatly simplify further development.

Not the answer you're looking for? Browse other questions tagged or ask your own question.