7

I maintain a tool that syncs files between a client computer and a document management server (using the CMIS protocol). Versions history (on the CMIS server) is important.

When saving a document locally, MS Word:

  1. Writes to a temp file ~wrdxxxx.tmp
  2. Deletes the original file Example.doc
  3. Renames ~wrdxxxx.tmp to Example.doc

PROBLEM: At step 2, my sync tool deletes the file on the server, loosing all versions history.

QUESTION: From the point of view of my sync tool, is there a way to know whether the file has really been deleted, or whether it is just being saved?

Notes:

  • Waiting is probably not a good solution, as I can't be sure how much time steps 2 and 3 take.
  • Cheking for the presence of ~wrdxxxx.tmp files does not work when several documents are being edited in the same folder at the same time.
  • The tool works like Dropbox: Users don't have to "commit" changes, files are synchronized automatically as soon as possible.
4
  • When / how often does your sync tool run, how is it triggered? Why does it run at a time at which the user is still working with the document?
    – Doc Brown
    Apr 30 '15 at 10:32
  • The metadata might be useful in re-establishing the historical links between the two versions.
    – rwong
    Apr 30 '15 at 11:04
  • 3
    Non-destructiveness of historical information is a design principle in version control. You just discovered that "sync tools" consist of a fundamental mini-requirement in terms of version control as well. If that looks like complicated to solve, it is a sign that "sync tools" are likely complicated to develop in the general desktop environment, where the user agents (software) are non-cooperating. The telltale signs of large software companies who tried to develop desktop file sync solutions (and be flooded with complaints) could be insightful.
    – rwong
    Apr 30 '15 at 11:12
  • @DocBrown: Sync might happen at anytime. It happens at regular interval, AND is also triggered by a FileSystemWatcher. May 1 '15 at 1:45
7

This is not "Word" specific, the same problem could occur when someone does this manually (make a backup of the file, rename or delete the original one, restore a backup etc.) Your sync tool cannot know which of these operations are creations of new documents, or just continuations of existing ones, and it cannot easily manage the version history correctly, since that would mean it had to "mindread" what the author/editor takes as "one" document.

Thus, all software VCS tools I know of expect the user to explicitly create a new revision, delete or rename the files explictly in the repository. If you are looking for a user-friendly solution, look how tools like TortoiseSVN or TortoiseGIT have solved that problem.

At step 2, my sync tool deletes the file on the server, loosing all versions history.

In case you still want to try an "automatic" approach, ignoring what I wrote above: when a file on the client is deleted, why is it really important to delete the history on the server? Can't you just mark it "deleted" as a special state, but still keep the history? When the file "reappears" in the next sync cycle at the same place on the client, you have to "revive" the file, undo the "deleted" state and continue the history.

4
  • When a file is deleted, it must be seen as deleted by other users of the server, and obviously many people use the server, not just my tool. Details: The server can be any CMIS-compliant server, I don't have the choice of protocol, I edited my question to reflect that. May 1 '15 at 1:50
  • @NicolasRaoul: I do not know CMIS, just took a look into the introductory pages after you added that information, and saw it supports a "deleted" state for an object, without destroying the history. Nevertheless I do not know if the CMIS protocol allows to revive a document later.
    – Doc Brown
    May 1 '15 at 5:48
  • I have proofread the specification several times and I am pretty sure there is no deleted state. Searching for "deleted" in the specification does not lead to any information about that "deleted state", would you mind telling where you have seen it? Maybe what you have seen is deleteType, which is only a way to delete a type, not a type that refers to deleted objects. Cheers! May 1 '15 at 6:00
  • @NicolasRaoul: you are probably correct, I was confusing the change log type "deleted" with the state of an object. But honestly, I still do not believe that "automatic versioning" without the user telling the system explicitly what he wants will ever work. As I wrote, the automatism would need to have mindreading capabilities for this.
    – Doc Brown
    May 1 '15 at 6:13
3

You have two things working against you:

First is a race condition, where the outcome of your sync operation depends on the timing of Word and that of your sync process. (If you're using something event-driven, the timing is a bit more predictable, but it just means that you're going to delete your history all the time instead of just sometimes as you would if you're polling.) As you observed, there's no way to know how much time will elapse between steps 2 and 3, so waiting to see if the file returns would create another race condition.

Second is uncertainty about whether a file you see as deleted will reappear because the OS can't predict what an application will do in the future. It also can't tell you what application deleted a file, which you might be able to use to draw a reasonable conclusion that the file might reappear.

If Word actually saves files in the way Microsoft describes, that's a flaw in the implementation. Windows has API calls to do atomic renames that would make the process go like this:

  1. Create temporary file ~wrdxxxx.tmp
  2. Atomically rename ~wrdxxxx.tmp to Example.doc

Because step 2 is atomic, your sync program would never see that Example.doc disappeared, just that its attributes or content changed.

Word's behavior isn't something you can change, so waiting is really your only option.

If the goal is not to lose version history, it's better to make deletions provisional and wait a fairly long time (say, 30 seconds) to make them permanent. This will prevent most -- but not all -- of the problems caused by not knowing how long a save takes. Obviously, it's possible for a save to take that long, but it's probably rare. The bigger pitfall is that a long enough provisional period will treat a delete/create sequence done by a human in that time as a version change rather than what it actually was. This doesn't reflect the reality of what happened, but it will preserve the version history.

The question only you can answer is whether having reality distorted like that on relatively rare occasions is acceptable in the face on not having your synchronizer blow away the history.

12
  • I'm afraid OP is merely coding directly to the Win32 directory change notifications, in the way of one notification, one sync. This simplistic approach is not going to work, and I don't know how OP is knowledgeably prepared or equipped for sophisticated workarounds.
    – rwong
    Apr 30 '15 at 11:42
  • @rwong: Whether it's polled or event-driven, the problems are still the same. The only difference is that events will cause a history deletion 100% of the time. (Just incorporated that into the answer.)
    – Blrfl
    Apr 30 '15 at 12:01
  • 1
    Sorry for bad wording; my "simplistic approach" refers to a strawman approach which is one that uses the Win32 events. What I meant is that it is not sufficient for the sync tool to react to each Win32 events in isolation, which I suspected the OP is doing. I did not mean to say your approach is simplistic or inadequate.
    – rwong
    Apr 30 '15 at 12:20
  • @rwong: No sweat. I didn't read it that way.
    – Blrfl
    Apr 30 '15 at 12:25
  • @rwong: Every time FileSystemWatcher notifies me, I check/sync the whole thing (all folders/files). I also run a sync at startup and every N minutes. Do you mean there is a more reliable way? May 1 '15 at 1:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.