2

We have a very large search service (written in golang if it matters) that runs on top of ElasticSearch, receive requests, builds the correspondent query, and returns the results (with some post-processing). The service handles thousands of requests per second and should be able to serve requests quickly and efficiently.

Since the service just READS from the database (writes are being handled outside of the service, asynchronous) and builds very complex queries who varies between users, requests and AB tests being run on production, most of the code in the service is code in charge of building the query.

Currently, the ElasticSearch query is being built mixing Business logic (AB tests running in production, classification of the user, etc..). In classic OO application with relational databases I'd probably use a DAL, the receives a request and know how to parse it to SQL, and then converts the returned value to something that the BL knows how to handle.

Building such a mechanism (an abstraction of our query, an intermediate language between our domain language query to ES query language) feels like an overhead. Since all of our service is really coupled to ElasticSearch, takes into consideration lots of ES features in order to increase performance, it feels like we'll just build our own objects who are replicated ES objects, and then do the mapping.

On the other hand, we think of the possibility we'll want to replace the package querying ES, or we'll want to update ES version and it will go through breaking changes, and we'll have to change a lot of code, scattered through the whole service and not centralized.

Any advice or help would be appreciated. Thanks.

Improve this question
1
  • I don't know if you can decouple the two, not really. You're relying on ElasticSearch to recover the data and pass it to you, likely with its own parameters to search with. However I would argue that having a layer of abstraction between your program and ElasticSearch is still a good thing anyway. It means if the ElasticSearch interface changes, your program changes very little.
    – Neil
    Jan 15, 2018 at 13:50

1 Answer 1

Reset to default
1

I also dont think that you need to decouple the two if your only concern is that ElasticSearch is updating the interface. If you have a decoupled layer then it should be more or less the same effort to change the code in the intermediate layer or in the coupled code.

A reason to decouple the code would be that you want to plug your BL to a new underlaying system such as (SolR, SQL,...) then you only would need to adapt the layer.

Alternatively, you could also look for an existing ElasticSearch Connector. There are connectors that could match to your BL. For example, you could use an SQL connector if your business layer is already writen in SQL.

Improve this answer
2
  • You can't assume the way you structure queries in ElasticSearch won't change between major versions. I've had to rewrite sections of my app for precisely that reason. I will say the changes between versions 1 and 2 were more invasive to my queries than the changes between 2 and 5 though.
    – Berin Loritsch
    Jan 18, 2018 at 14:23
  • @ Berin Loritsch, thanks for the feedback, I removed that assumption.
    – Rene B.
    Jan 18, 2018 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.