0

I need to make a program to compare several large +100MB XML files, find the difference, and spit out JSON. I have written the comparison code in C++ and it performs great but now when I am getting to the bit about working with the XML data and converting it to JSON I find I'm hitting a lot of walls.

Would it be logical to port the XML comparison code to NodeJS, which may perform slightly slower, but make things a lot easier when it comes to converting XML to JSON and modifying the contents?

Any input on this would be appreciated.

James

EDIT: Or alternatively, should I keep the computational side of things as a nodejs module in C++?

Improve this question
3
  • 3
    According to benchmarks, JavaScript on V8 can be up to 10 times slower than C++ (worst results). You know how much time it takes to perform a task with C++ implementation. If you multiply this number by 10, would it still be acceptable time?
    – scriptin
    Commented Apr 19, 2015 at 10:36
  • @scriptin The quicker the better, 10 times would be my limit. I just can't really seem to work well with xml and json in C++ whereas I can with Python or node
    – Jimmy
    Commented Apr 19, 2015 at 10:38
  • 1
    This question IMHO does not make any sense, wondering why it did not get any downvotes so far. If you have already written C++ code for comparison of XML files, and it performs already great, this obviously includes the hardest part of working with XML, the XML parsing. It is not understandable why writing the results to JSON should be so much harder than the task you already solved in C++ that you are considering to rewrite everything in another language.
    – Doc Brown
    Commented Apr 13, 2017 at 12:47

2 Answers 2

Reset to default
3

IT depends on the frequency of the task in your hand. If it is going to be one time job, or let's say once in a year job, do not worry about the speed, reduce the complexity of your code (independent from the language you are planning to use).

Otherwise, if such XML files are created every day / hour and require JSON conversion, I will even prefer C since standard libraries might be troublesome.

NodeJS is good for network applications, on the other hand, your problem seems like a good old CLI application. For Interpreter languages, you might want to try Ruby or Python.

A quick search might even give you good libraries or open source implementations.

Do not forget to sample your data for testing.

Improve this answer
6
  • Thank you for the help. I need to run this every few hours each day with a lot of data. I'm having a really hard time working with xml and JSON in C / C++. Python was running a bit slow for my liking (at least 15 times slower)
    – Jimmy
    Commented Apr 19, 2015 at 10:20
  • 1
    Is your xml source external to your system? You might want to try a conversion earlier when your system receives the data, or you might want to use an opportunistic deamon to utilize conversion in parallel.
    – Umut Kahramankaptan
    Commented Apr 19, 2015 at 16:17
  • Yes it is external. Are you thinking to convert it to json straight away and then process that, or did you have something else in mind?
    – Jimmy
    Commented Apr 19, 2015 at 16:19
  • 2
    Now I am thinking, could you describe the system and your need a little bit more, if it is not sensitive. I can suggest a more solid answer if you can support the requirement and background of the domain.
    – Umut Kahramankaptan
    Commented Apr 19, 2015 at 16:19
  • I pull in a fairly large number (10 initially, around 50 in the future) of 100mb xml files from web domains, then I work out the difference between the files, i.e what nodes have been added / removed in the newer file, then I pump out the results in JSON. I can't vary my output format (json) or my input format (xml) but I can do whatever I want in the middle. Since this is run in the cloud time is money.
    – Jimmy
    Commented Apr 19, 2015 at 16:22
-3

I would use C# because of the .NET framework. It provides lots of good built in libraries for XML and NewtonSoft JSON is a good NUGET package you can add that will reduce your code and make it very simple.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.