3

At my new work, a few people want to share modules between systems/apps, and I'm a bit skeptical about it.

Context: we have a lot of little apps living in different servers, some of them are scrappers, some just web apps etc... they are different languages too (java, javascript, php, python) maybe the only thing that they have in common is the export to csv/xlsx for different clients.

So the idea was to create a shared "export module" that every system/app could use.

The new workflow is that they want to create is:

  1. scrape/process the data
  2. store this info in the db (make sense to me)
  3. then with a unique module create the "output" that right now every app is creating at their own.

The last part is where I'm insecure, the part where a unique (maybe nodeJS) module tries to get right output for different system with different business logic, for example some scrapers create a file with the last Monday from the week in the name, another one just create a csv file with "today" in the name.

Is possible to share a module at that point, or is it better if every app is independent and atomic?

Summary: two options

1) Multiple systems in different languages store the data in the db then a unique module create the output from them

2) Every system with his "own" library to export to csv, to excel, etc... as needed? (still saving the data in the db)

  • Do yourself a favor and make a shared module that is also flexible and can be reconfigured easily. In this way, it can start off as being a shared module, but you can easily adapt as needed. – Neil Sep 11 '18 at 13:46
  • I have posted an answer, I am sure others will as well. Perhaps you can edit your question based off of my answer if you feel I haven't completely answered it. – Jeff Sep 13 '18 at 13:43
3

One principle to consider is that if you are writing in multiple languages, you will have to find a team that can support the multiple languages. Not the largest concern, but worth mentioning. I've worked in places where I wanted to use golang for a small microservice, but it was decided to stick with Java since we had a number of Java experts on staff and finding a backfill for that position would be easy.

The first question I'd ask is, "What problem is being solved here?" It sounds like what they want to do is have one export module so they can DRY (don't repeat yourself) up the code. That is a good thing to do; it helps with maintainability. But, what is the biggest problem here? Is it maintainability?

Additionally, how different are the data structures that need to go to the customer? How are the customers accessing the data (in the same way, or in different ways)? I think these might also point you to a decision.

Microservices may be a good paradigm to consider here. Every app could be it's own service and the "ETL" microservice could load the data into different database tables. Different web endpoints could access the different tables.

Ultimately, I think the problem statement needs to be thought out before making a decision, but either approach could be useful.

  • Hi thanks for your answer i really understand your point, but you don't think that a app/system should be atomic? because when i see for example an app in c# that do everything right but depends on a external module in another programing language to export his files, it feels weird – GVB Sep 14 '18 at 14:20
  • Well, think about libraries. Some developer wrote a lib, you imort that into your code, I import it into my code. They update it, we both benefit. You could essentially be writing library code that all your projects can use. – Jeff Sep 14 '18 at 15:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.