10

I want multiple users to edit same document. Problem I'm facing is when a new user joins, he might see an outdated document. How do I make sure that new users get most recent changes?

Some solutions I thought of:

  • Save on every change. I don't like this solution because it will slow things down on UI and put load on db.

  • When new user joins, trigger save on all other clients. After other clients saved, load document. With this there can be inconsistency still.

Any other suggestions would be helpful.

UPDATE: After looking into suggested solution, Google Realtime API, I found out that:

  1. Users of your app must have Google Drive and give you access to their drive. This could at best present awkward UI flow or prevent users who don't have Google Drive from using realtime feature.

  2. All the sharing settings that are done on your side, have to be replicated for Google document.

UPDATE 2: To acoomplish the goal, I went with Google's Firebase

  • Why is there a difference between a new user and already active users editing/viewing the same document? – Andy Jul 18 '13 at 20:34
  • @Andy What I'm currently doing is broadcasting via sockets all the changes users make. These changes update UI for users who have their browsers open but they are not instantly saved to database. So I have a situation, when a new user joins, he loads document from database and he doesn't see all recent changes that weren't yet saved. – dev.e.loper Jul 19 '13 at 15:33
  • 1
    if you already sending changes, and want to leave same behavior like it is now, you can ask one of clients send last view to new client or you can have one virtual client on server, who gets all changes and when new client joins sends latest view to it. – Dainius Jul 22 '13 at 14:00
14
+50

Google Drive

If you're trying to make your own version of google docs, I suggest you take a look at the Google Realtime API. Google recently released this with the intent of allowing other developers to use the same tools they did to allow for realtime collaboration. This would allow you to save time on your development and get a working product sooner.

You could easily take the data that is in the document and push it into your database at regular intervals, or have the database itself be a 'participant' of the exchange, simply listening to and logging all the changes. It also allows for a user to define their own data structures which are then usable in the realtime API, so you are free to extend it as you see fit.

Non-Google Drive

So according to your research, Google Drive isn't an option. That's fine, but it's going to be tougher and possibly not work as well, depending on how much you put into it.

Here's a general strategy I would use to accomplish this problem:

  1. Have the server be the communication multiplexer. Each person talks to the server, and the server sends out that information to everyone else. This way the server always has the most up to date view of the document.

  2. Find a third party algorithm/module for conflict resolution. Conflict resolution is tough, and is something that still isn't perfect. Doing this alone could easily increase the scope of the project to be far too large. If you can't use a third party algorithm, I would suggest that you only allow one user to edit an area of a time, so that the user must obtain a lock before editing an area, or you risk destroying another users work, which will get very old, very fast.

  3. When a new user joins, give them the most recent document and automatically start streaming the commands to them. The server has the most recent view and thus can dish it out automatically.

  4. Backup to the database at certain intervals. Decide how often you want to back up (every 5 minutes or maybe every 50 changes.) This allows you to maintain the backup you desire.

Problems: This isn't a perfect solution, so here are some issues you might face.

  1. Throughput of the server could bottleneck performance

  2. Too many people reading/writing could overload the server

  3. People may become out of sync if a message is lost, so you may want to make sure you synchronize at regular points. This means sending out the whole message again, which can be costly, but otherwise people might not have the same document and not know it.

  • Yes, the changes are broadcasted to all the clients and they have their (hopefully same) version on the browser. Sounds like you are saying that, updating document on every action, is a way to go? – dev.e.loper Jun 26 '13 at 13:34
  • Or at least have regular 'sync' timeframes where the current state of the document is transmitted out in the background to make sure that everyone is on the same page. How often would depend on how fast people would be changing the document. That way you already have an established method for sending to new people, as well as the ability to make sure it never diverges too much. – Ampt Jun 26 '13 at 13:36
  • 1
    +1. Don't make life difficult. Google does this well without having to reinvent the wheel. – Neil Jun 26 '13 at 13:56
  • Does Google Realtime save to Google Drive? I want to save to my database, not Google Drive. – dev.e.loper Jun 26 '13 at 13:58
  • @dev.e.loper added some info about that to the answer for you. – Ampt Jul 2 '13 at 18:42
3

I'd recommend 1 persistent copy of the document on the server. When a client connects to the server you issue an UPDATE command(s) to that client with all the changes.

Update WorkFlow

User causes triggering change -> Client sends UPDATE to Server -> Server sends UPDATE to Clients

Viable triggers

  1. User clicks Save
  2. User completes a specific task
    • Finishes editing a cell
    • Finishes editing a sentence/paragraph/line
  3. User clicks Undo
  4. User presses Return key
  5. User types a key (save on every change)

Update Implementation

I would suggest being able to re-create the document with a series of UPDATE commands so that the server stores each UPDATE and when a new client connects the client can be sent the series of updates and it itself can re-create the document to display to the user. Also, you could alternatively have a SAVE command that is separate and have UPDATE be temporary changes that can be used for UNDO requests and have SAVE actually store it to be re-opened if the server is closed or all clients disconnect.

  • 2
    What about conflict resolution? What if two people edit the same area of text at the same time? Also, this seems to place a load on the DB, which is something OP was looking to avoid. It might be viable for what he needs though. – Ampt Jul 2 '13 at 18:09
  • @Ampt I made a spreadsheet using this model, and for conflicts each specific task being updated was completely replaced by the newest version. So the last person to finish editing a cell would replace the previously updated one completely with no merging. – Korey Hinton Jul 2 '13 at 18:11
  • 1
    So one sentence would overwrite another one if this were, say, a word document? – Ampt Jul 2 '13 at 18:13
  • @Ampt yes, alternatively you could implement a way of locking what is being worked on, but I took the easy route. – Korey Hinton Jul 2 '13 at 18:16
3
+50

1) Have a look at Knockout.js

It follows an MVVM pattern and will automatically push out notifications to the View based upon changes to the Model. For instance, look into their observable array to provide a little more information on how they do that.

2) Mix that in with SignalR and you should now have the ability to send out notifications to other users working on the document. From their site:

SignalR also provides a very simple, high-level API for doing server to client RPC (call JavaScript functions in your clients' browsers from server-side .NET code) in your ASP.NET application, as well as adding useful hooks for connection management, e.g. connect/disconnect events, grouping connections, authorization.

So you'll need to have some hooks at your model level within Knockout.js to make some SignalR calls whenever a change occurs. The other clients will receive the notice from SignalR and then trigger a corresponding change in their copy of the Model, which will push back up to their View.

It's an interesting combination of the two frameworks, and you should be able to search and gather more information to handle the particulars.

For example, this codeproject example specifically addresses Co Working UIs and Continuous Clients which seems to be exactly what you're trying to do.

New age web applications may need to offer new age user experiences - and should handle co-working and continuous client scenarios properly. This involves ensuring that the user interface is syncing properly itself across devices and across users to ensure the state of the application and user interface is maintained "as is".

This blog post looks to be an entry point into a series of blog posts discussing the use of the two packages and contrasts that with a traditional ASP.NET approach. May provide some points for consideration while you're designing your site.

This blog post appears to be a little bit more basic and provides the groundwork for combining the two packages.

Disclosure: I'm not affiliated with any of the above links, nor have I really dug into their content to see how sound or correct it is.

2

The solution is Operational Transformation (OT). If you haven’t heard of it, OT is a class of algorithms that do multi-site realtime concurrency. OT is like realtime git. It works with any amount of lag (from zero to an extended holiday). It lets users make live, concurrent edits with low bandwidth. OT gives you eventual consistency between multiple users without retries, without errors and without any data being overwritten.

But implementing OT is a difficult task and time consuming. So you might want to use an external library like http://sharejs.org/.

  • 1
    Google Realtime API is doing OT youtu.be/hv14PTbkIs0?t=14m20s They do it on both client and server. I couldn't get a clear answer from reading ShareJS docs but I'm assuming that ShareJS does OT on both client and server? – dev.e.loper Jul 22 '13 at 13:04
1

It mainly depends on the type of your documents and how your users collaborate.

However, I would:

  1. let all clients send unsaved changes to the server every once in a while (depends on how users work with the documents).
  2. the server stores the deltas in the user's session (even for a fat client you need something like a session)
  3. other clients editing/viewing the same document get those temporary changes or at least a hint that there might be so.

Advantages:

  • no DB updates unless someone clicks 'save'
  • backup for the case the client crashes (for the session-period)
  • your server decides how and what data to forward to which client (e.g. you can jump start the feature with just a note and later implement a more sophisticated merge and highlight)

Disadvantages:

  • not 'real-time' - e.g. you send every 30 secs, but someone types 3 sentences in that time.
  • more network traffic - dependent on your documents and collaboration
  • possibly large sessions
  • possibly high computation effort if many users collaborate and do many changes
1

Essentially, what you are asking is how to deal with shared mutable state. Saving is the easy part; but how do you deal with multiple people editing the same thing at the same time? You want all users to be viewing the same document while synchronizing simultaneous edits, all in real-time.

As you have probably gathered, it's a hard problem! There are a few pragmatic solutions:

  1. Modify your application requirements to not allow true simultaneous editing. Edits can be merged like with source control systems, with results broadcasted to each client. You could build this yourself but it'd be a poorer user experience.
  2. Outsource the synchronization of state mutations to an open-source solution that integrates with your existing technology. ShareDB is the current leader in this space. It is based on Operational Transformation and used in at least one production system. This will take care of the saving issue you're concerned with but won't help with any of the additional UX features mandatory for any collaborative application.
  3. Use an off-the-shelf platform such as Convergence (disclaimer: I am a founder) to handle all the difficult bits for you. You'll also get additional tools for real-time collaboration such as cursor/mouse tracking, selections, and chat to build out a superior collaborative experience quickly. See this question for a good roundup of all the existing tools out there.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.