0

There is a remote csv file that gets updated every second or so.

I need to download it(on a Windows machine) ONCE and always sync local file with the remote one.

Obviously, downloading the whole file every time is not an option. I need to download only the changes.(something like rsync, rdiff-backup)

I searched quite a bit but could not find how I can do this.

I am sort of new to nodejs and am using this app as an opportunity to expand my nodejs skills. Also, I am planning to use nodejs and to package it using node-webkit(https://github.com/rogerwang/node-webkit)

Improve this question
8
  • 1
    This can only work if on the remote side some process provides you the changes in a separate file (for example, by running diff) or a bunch of files with timestamp for each change interval. The question is: are you free to choose what runs on the remote side?
    – Doc Brown
    Jun 29, 2013 at 7:17
  • By the way, the core of the question has IMHO nothing to do with javascript, node.js, not even with Windows, so this question seems to be mistagged.
    – Doc Brown
    Jun 29, 2013 at 7:21
  • @Doc Brown I want to do this in a Javascript/Nodejs app using Javascript and Nodejs ... that is why tagged it as javascript, nodejs
    – Amit Dugar
    Jun 29, 2013 at 7:29
  • I have no choice or control on remote side ... they generate a CSV using some other means .. I need to provide my windows client with an updated CSV file ....
    – Amit Dugar
    Jun 29, 2013 at 7:30
  • 1
    If your server side does not provide you with a diff mechanism, what you want is technically impossible. Finding out differences between two files needs to read and compare the files, and for doing that on the client side one needs to transfer the whole data over the net first. And this problem is independent from the language you want to solve it with. What matters here by which protocols your server provides you with the file (ftp? http?)
    – Doc Brown
    Jun 29, 2013 at 7:54

1 Answer 1

Reset to default
2

As the comments say, this is impossible without the remote server providing some sort of native diff.

I see two options

  • Download the whole file and do a diff yourself - Potentially expensive, but works out of the box without requiring the remote server to add diff features.
  • Add support for an update stream on the remote server - This would allow you to download the file once and then somehow subscribe to updates. The best way to do this entirely depends on the protocol which you have not specified. It could be as easy as running rsync or as complicated as a separate CSV file that has entries specifying the row, column, and the new value since the last fully generated main CSV file.
Improve this answer
1
  • I understand that it is impossible, but I was hoping there was some magical way to achieve this :) I will have to explain this to the client and try to convince him to get the other dev to give us an update stream ...Thanks
    – Amit Dugar
    Jun 29, 2013 at 17:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.