0

I am tasked with making a program that will roughly have the same steps as a program that a former has written. The program that I'm trying to imitate doesn't seem to be open for extension, otherwise I wouldn't be writing another similar program. Below are the steps in the written program with the corresponding technologies used for each step:

  1. Make a GET call to a REST API - VB
  2. Clean junk from an XML file - VB
  3. Zip the clean XML - Powershell
  4. Process XML and store in DB - MSSQL stored procedure
  5. Use a vendor file transfer system to transfer processed data from one DB to another - Powershell passes the data to the file transfer system

There is probably a reason why it was written that way, but nobody in the company knows those reasons. There were talks of rewriting everything into MSSQL stored procedures, but this seems like we would be using a tool that wouldn't fit the task. I would replace the steps that use Powershell with VB (or any other programming language) and leave MSSQL for storing and retrieving data from the database.

What would be the best practice in a case like this? Is there any general agreed upon convention or rule of thumb that I can follow in the future with some kind of reasoning behind it?

Edit: To clarify what I meant, I need to write another program that is similar to the one I described above. The functionality is around 80% the same. I could just use the same execution flow, using the same languages, and just add that 20% that I need to write on my own. The program described above doesn't seem to be opened for extension since it wasn't written with extensibility in mind. My question is, given the steps and technologies above, what would be the most optimal way of choosing the languages?

5
  • "There is probably a reason why it was written that way, but nobody in the company knows those reasons" I think your two statements are related and answer each other. Sounds like a bunch of clueless people were tasked with doing some patchwork system. I don't even mean "clueless" derogatively - it might very well have been tasks that seemed unimportant at the time given to different people without explaining the full picture and then later these were glued together. It might be a legacy project ferried through multiple people before each left, too. It's not really a new phenomena. – VLAZ Mar 15 at 12:30
  • 1
    "doesn't seem to be open for extension" is pretty vague. Do you have an actual requirement to change something in the system? At which at the 5 steps? If the system works for now, and there is no real change necessary, the "best parctice" is not to touch the system (now). – Doc Brown Mar 15 at 12:42
  • Depending on what is involved in "Clean junk from an XML file", I'd recommend using XSL for that step. – FrustratedWithFormsDesigner Mar 15 at 15:42
  • 2
    There is nothing like "a most optimal way of choosing the languages". Which language you choose will depend on what you know best, if the language is powerful enough for the task (VB.Net and Powershell will probably be), if both languages can be expected to have support for >10 years (will probably the case here), how much of the existing code you can and want to reuse, and several other factors. – Doc Brown Mar 15 at 19:36
  • 1
    I'm with Doc Brown-- choice of technology comes down to your team's competency and your organization's roadmap for the future. That being said, there are tools for converting both VB.NET and PowerShell to c#, so if you're good with that language, you'd have a decent head start. It'd much more difficult to convert them to, say, Java. – John Wu Mar 15 at 21:26
4

Optimality requires a goal.

And goals aren't the most stable of things.

For example, lets say your goal is to implement this in the quickest time, and lets for he sake of argument measure this by total hours on the clock from... hmm.. now.

So you throw up some quick design ideas:

  • Duplicate the current code and modify it
  • Pick an Enterprise™ platform and reimplement it in it
  • Pick a language, development environment, operations platform and rewrite it.
  • Hire 12 programmers to re-implement it.

Now which one is the quickest? Every second you spend now is just making the quickest solution take more time. But how to decide?

Hiring 12 programmers sounds great. After-all any solution they go with will take 11 times shorter right? Probably not, they need time to form working relationships, get used to their new tools, consider this problem from end to end (which is something you've already done), not to mention the time spent hiring which can take months.

Okay, What about a new language? Well that is going to take time to explore, to choose between various options. You are going to have to learn a few to enough skill level to be able to judge the time needed to implement something within it. Not to mention picking the programs to form the development environment. You might even need to consider how this will be deployed. Wow that is a lot of time and decisions to make before you even start writing the solution.

Surely big Enterprise™ have already solved this. Why yes I believe they have with Data Manipulation Program 234123, just pay 3 trillion dollars here and in 2.6years you can have 80% of the functionality. Hmmmmmmm....... Do you really trust that salesman.

Okay, lets do this directly. Duplicate all of those programs and rename them _2 (please don't do that, pick something meaningful). Now just change the few lines of code to get the desired functionality. Except the reason you want a better solution is probably because:

  • The current code is unsupported
  • You don't have the inhouse skills to do this easily
  • The system probably won't meet a perform requirement once duplicated

So no that is going to take time to reunderstand someone else's thinking that was probably half-baked, highly distributed between non-communicating individuals, or in a paradigm you can't quite grasp.


Values

Sarcasm aside, the problem isn't even that you don't have a goal. You're pretty clear about wanting to rewrite. Its that you don't appear to have your Values in order.

What you value identifies what goals you have which in turn highlights what is optimal.

For example a Business usually has conflicting values:

  • Biggest bang for Buck
  • We want to exist in 5 years time

In pursuit of the biggest bang for buck goal, you will value fast coding over long term coding.

In pursuit of the we want to exist in 5 years time goal, you will value clean and well managed code over quick changes.

Not saying you can't have clean well managed code that supports quick change. But you do need to know, when the chips are down, which goal, and hence which value wins. Does the change get delayed till its good? or do we rush the change out, and try to cleanup the mess while also delivering the next change?

This also applies to your question about what is the best way to handle your new functionality. What is valued?

  • Does your organisation value a simple copy and extend over a rewrite?
  • Does your organisation value keeping the language and tooling standard for its developers, or does it value many niche language systems?
  • Does your organisation value simplifying the operational technology stack, or do they value not fixing what isn't broken?

When you know what the answers are to those questions, and questions like it. You will be able to whittle down the field of options quite drastically.

1
  • This is a really good answer, I think it could be given to many questions that were asked on this site. – Doc Brown Mar 17 at 6:44
0

You should move the architecture toward something database centric, then the languages and libraries will be totally separate for each process, and each can be independently replaced in the future as programming-language-trends change.

You can keep the legacy code and only "upgrade" it as needed. You can do one or all of the following as needed over time:

5. Transfer data from one DB to another

That's a background process that is triggered when there is new data (or on a timer). Ideally, the process is on the destination side and reads (SELECT) from the source as needed. It has a dedicated read-only database account (username and password) and secure network connectivity.

The Passage Problem - The network connectivity part has traditionally been the hardest, and this has led to poorly designed workarounds, such as the use of SFTP with CSV. Make sure you overcome the passage problem head on. Then you can probably even use an off-the-shelf tool (like MS SSIS even), although I personally prefer a console application (whatever language you like)

4. Process XML and store in DB

This step is eliminated with [3].

3. Zip the clean XML

Plenty of languages have simple support Zipping. This is a background process waiting for a cleaned-XML record that doesn't yet have a matching ZIP record. This can be written in any language you like. This will write the result into a table of zipped records.

2. Clean junk from an XML file

This is a background process waiting for a new XML record that doesn't have a matching cleaned-XML record. This background process can be written in any language you like. This will write the result into a table of cleaned-xml records.

1. Make a GET call to a REST API

This is a database gateway. Simply write the record directly to the not-cleaned-XML records table. Even better, you can use GraphQL instead (and other similar general-purpose database gateway tools so you don't have any custom code to maintain)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.