0

I work with a integration project based on Publisher/Subscriber pattern whose subscriber end feeds a staging database which stores data to be pushed to another application. The staging database has some views which are responsible for providing data the the final application

We use DataDog for observability, but since no part of the integration system call those views, we aren't notified when something goes wrong with them. Last week, one of the views was changed and we only were notified by the final application user.

So my question is: is it valid to create unit tests on the subscriber part of the integration system to check if the views are ok, even if any piece of code calls those views? Currently, the responsability of the subscriber is store data on the staging tables. It has any interaction with the views. Even the suscriber's DAO layer hasn't methods to query them.

I'm wondering if i'm not going beyond the integration system frontier and ending up testing things that are not itegration system responsability. Or should I say "screw the theory and solve the problem"?

Im open to suggestions on other approaches.

2
  • "but since any part of the integration system ..." - I guess you mean "but since no part of the integration system ..."?
    – Doc Brown
    Commented Mar 10 at 22:16
  • @DocBrown, sorry, English is not my first language. Edited and fixed. Thanks. Commented Mar 11 at 11:00

1 Answer 1

1

The staging database has some views which are responsible for providing data the the final application

It seems to me that you have exposed the database without an API. If you had to access those views via an API then you would have tests for the API methods which would highlight the errors you experienced.

There are a number of approaches If you don't want to add an API layer to protect your database:

  1. Maybe your "final application" has a repository or api layer which uses the database. By moving this to the subscriber project instead and consuming it as a library you could add tests to this component.

  2. You could add an API layer directly to the database via Stored Procedures. You can then add tests for those procedures

  3. If you have database tests for the sprocs, you might as well add some for the views.

4
  • I don't need an API because the final application is a third party product which pulls the data through a simple ADO connection. If you look at the first image on the MS article I mentioned, I'm the Subscriber box. My Subscriber app push data to the staging table. It is its unique job. The third party application will read data trhough the views. In summary, my question is, should I develop unit tests for those views or it is responsability of the third part app?? Commented Mar 12 at 19:09
  • its unclear what's going on. If you are inserting rows directly into a third party db, then you shouldn't, the third party should provide an API. If the third party is pulling directly from your DB, then it shouldn't. You should provide an API. and test it.
    – Ewan
    Commented Mar 12 at 22:32
  • if the third party is pulling from your DB and you cant make it use an API, then the view IS your API and you should test it
    – Ewan
    Commented Mar 12 at 22:33
  • I'm not inserting rows into a third party. I only insert rows in a staging database. The third party that pull rows from it. Unfortunately the third party application doesnt provide any other way to receive them. It is a black box hosted by the vendor and we cannot change it. So I cannot make it use an API as you suggested. Anyway you answered my doubt. Thanks. Commented Mar 13 at 11:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.