3

From what I understand, no-SQL databases differ from SQL databases because they allow the developpers to designe the tables to fit the usage, rather than fitting the model. Which means most of the time that if I have an API which uses Cassandra or Neo4J, Cassandra or Neo4J will have one view per API service.

If this is true, then does this mean that if I have a Spark application that fills my no-SQL database, what team in the project should in theory be responsible for Cassandra or Neo4J ? By responsible I mean: creating the tables, defining the table creation standards, delivering the database and the tables to the client... Should the Spark team take care of that because it is the ine filling the table, hence owning the data, or is it the API team because the schemes are designed to feat their application? I guess they will have to communicate, but who would perform these tasks ?

0

Under my point of view, the scenario you mention is a common situation in companies out there.

An API service might be designed by a Software Engineering Team following certain requirements, having already a database there, so the API, a later solution to make use of this data.
It could be also designed at the same time the API service was built.

For big projects, the database itself has a database guy (or team) which is expert in the database technology, highly aware of tricky technical details, enough to make design decisions or provide a robust, optimal database system.

If we are talking about data science databases, mainly fill from calculations made on data (the spark for example), maybe the team who knows best what to do with the database is the data engineering/data science team, and the API team, this time, responsible of creating business logic to "help" on any kind of query, post-process or complex reporting from this database.

There can be also the case where the database is more like a collection of object oriented registers, to be created/read/updated/deleted, by for example a REST approach to the API service, then, it is more likely that the API team manages completely the database.

I agree with @Ewan that it depends on your company makeup, how the system got built, who designed what, and how the experts are aligned within the teams.

  • 1
    "The team who knows best the data" could actually be a solution. In fact, this rythms with "The solution that will cost the less energy over time" which might be the decent answer. – Vulpo May 10 '18 at 15:55
5

The API should completely hide the database. Regardless of SQL vs No-SQL approaches.

That means that:

  • The DB should be filled via calls to the API.
  • Callers of the API don't have to know what the database implementation is

Which 'team' is responsible really depends on your company makeup. But the design allows the API developers (perhaps this team includes DBAs?) to maintain the DB as part of the API. Rather than the old style of having a central DB used by many applications, where the DB Views and Sprocs were essentially its API

  • In my case the API only reads the database. The database is filled with a spark application which calculate the data to be put in the DB. So who should maintain the database scheme ? – Vulpo Apr 27 '18 at 10:51
  • 1
    add the populate calls to the api, then you have a single team which is responsible for a database design which supports all requirements, rather than two teams with competing requirements – Ewan Apr 27 '18 at 10:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.