2

Background

I have a system that comprises of 3 components:

  • a web application that displays data and allows data manipulation by users (React)

  • a server application which serves the data to the web application and handles the manipulation requests (Node)

  • a command line application which builds said data (it takes input from the user, builds the data structures and sends them to the backend to be saved) (Python)

The e2e flow typically looks like this:

  1. I start the command line application and it uses the input I give (may be a file) to construct the data structure and send it to the server application

  2. The server then saves it into a database

  3. I then open up the web app to visualise the data, it fetches it from the server

  4. I may want to modify some parts of the data using the web app, if I do a modification the web app will update its own version of the data structure locally (in memory)

5.a. If I want to save the edits I press on a Save button and the changes are sent to the server

5.b. In some other cases the changes are sent automatically (they are also applied locally, in memory)

Problem

There is a lot of commonality between the way the data structure is firstly built and the way it can be modified by the user. These common operations range from simple to very complex so there is severe code duplication between the command line app and the web app. I'm worried that I won't be able to keep them fully in sync (and I had issues caused by this before).

Solution

The obvious solution seems to me to move all the common functionality inside the server application.

Problems with this solution

I am worried about the case when a user tries to do a modification via the web app.

Currently what we do is we apply that modification in the web app itself first then we subsequently send the updated data structure to the server to be saved. The benefit with that is the user instantly sees the updates being applied. The drawback is if the update cannot be persisted for some reason, the user either still sees the updates being applied (before they refresh) or they see the update being rolled back. In my case, the benefit outweighs the drawback so I'd like to keep it this way.

But if we move the code that actually does the update to the server app, the web app will have to wait until the request is completed before it can display the update to the user (correct me if I'm wrong). Since that update can be a very complex one (and the data itself is very complex, we have graph and tree structures), it can potentially take a long time.

Questions

  1. Is the obvious solution the most appropriate one? Are there other ways to avoid that kind of duplication?
  2. If this is the solution, is there any way I can improve upon this solution or will I just have to live with it?
  • Is there anything blocking you from rewriting the web app into a Python framework, such as Bottle or Flask ? Since the most complex part of the job seems to be the command line application, maybe rewriting the web app using Python may be the best code-reuse approach. – Machado Dec 6 '19 at 13:31
  • @Machado it seems like you were confusing the web app with the server app. That's exactly why I put the techs I'm using for each system in parantheses, but maybe I should've put some more effort into making this unambiguous. Thanks a lot anyway. – Andrei-Marius Longhin Dec 9 '19 at 10:41
3

Here is a different idea which might work for you. Instead of moving the code to the server,

  • isolate the code in the web app you want to reuse from the command line application and put it into a Javascript module

  • make that module callable from your command line app and reuse it there directly.

The second step is probably the hard part. You could either do this by rewriting the command line app completely in Javascript / node.js, or by using something like python-bond or PyV8.

(Disclaimer: I haven't tried this by myself).

See also: Executing Javascript from Python (on Stackoverflow)

| improve this answer | |
  • Or do the opposite, making the web application reuse the code from the command-line app. – Machado Dec 6 '19 at 13:31
  • @Machado: what you suggest means to put the reusable code on the server side of the web application. That is essentially what the OP already suggested by himself, with the described drawbacks. – Doc Brown Dec 6 '19 at 13:49
  • Indeed, but in Python. It may be obvious at first glance, but maybe rewriting the web app in Python would be less work instead of rewriting the complex command-line in Javascript. // Edit: Maybe adding an application server in the middle, using something like an Actor model could be a good solution as well. – Machado Dec 6 '19 at 13:53
  • @Machado: my suggestion does not necessarily require to rewrite the command line app in full. And I have no idea why you think the command-line app is more complex than the web-app, as far as I can see the OP simply did not tell us. – Doc Brown Dec 6 '19 at 13:55
  • You're right, there's no statement of that in the question. But since the command line app generates the data and also updates it in the backend, and has duplication along with the server, I'm assuming it's the critical path of the code. If this premise is correct, which we don't know, I'd go with rewriting the web app, if it's feasible. A lot of "ifs" to be considered. – Machado Dec 6 '19 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.