How to keep front-end models in sync with REST API/DB without any UI delay?

Question

Here's my problem. I want to make fast, subsequent requests to a REST API endpoint (/todos/:id/position) that have side effects on other resources.

As an example of 5 fast subsequent requests:

PATCH /todos/2/position {"position": 2}
PATCH /todos/2/position {"position": 3}
PATCH /todos/2/position {"position": 4}
PATCH /todos/2/position {"position": 5}
PATCH /todos/2/position {"position": 6}

As you can see, I am moving todo #2 from position 2 to position 6, one request at a time. Now of course, those calls have side effects on other todos. When todo #2 changes from position 2 to 3, the position that had position #3 goes from 3 to 2, etc. This provides some sort of race conditions, as the 5 calls are modifying data in the DB at the same time.

I have implemented transactions and ETags on this service, because at first I thought I was having concurrency/optimistic locking problem, but it seems like my problem is bigger than this. With ETAGs and transactions in place, request 1, 3 and 4 might works, but requests 2, 5 might fail, resulting my todo to only reach position 5 (and also awkwardly skipping over certain other positions). This is because request 2 does not have the right etag, because request 1 has not responded yet, etc.

How can I make sure all requests are successful and execute in the right order? Is this even possible?

Here are two solutions I have thought of and that I consider weak, since they do not resolve the root problem on the server.

1. On the front-end, have a queue of requests. Request 2 is not initiated until request 1 has resolved. Request 3 is not initiated until request 2 has resolved, etc
2. Throttle the requests. Instead of sendin 5 fast, subsequent requests, throttle them to only send the last one, i.e with a timer of 500ms.

Answer 1

You really only have a few options on how to handle something like this. The first is to change the API so that it accepts batch operations. If you had something like this:

POST /todos/position/$batch
{ 
  "operations": [
    {"id": 2, "position": 5},
     ....
  ]}

You could apply them all or rollback at one shot.

Another option is to change the client to do compensating transactions when there is a failure. This is on the client and does not guarantee strong consistency, but if you have a position change resource verb, your client could track the successes and send follow-up transactions to roll them back to the previous state. This idea is covered very well in Sagas by Hector Carcia-Molina and Kenneth Salem (https://dl.acm.org/doi/10.1145/38713.38742) and far predates the RESTful architectural style, but the idea is still there.

Finally, you could implement full two-phase commit in your REST API which is probably the hardest and least RESTful option you have.

I will probably be roasted for this part, RESTful architecture is fundamentally based on viewing your system has objects (resources, documents, etc.) but I find this doesn't always offer the lowest amount of friction when mapped to business processes. It is often more pragmatic to compromise a little bit on what makes things RESTful than it is to remain dogmatically pure about this architectural style.

Answer 2

Possible UI solution:

Block the UI interaction for that element, as long as your client app waits for the API response.

That is in my opinion much better then having a queue or threshold for subsequent api calls.

Possible API solution:

(Depends on your client app, if you can drag'n'drop your item)

Don't update each todo item as a single resource, instead add a todo list resource, which would return you this

{
  "todos": [
    {"id":1, "position": 1}
    {"id":2, "position": 2}
    {"id":3, "position": 3}
    {"id":4, "position": 4}
  ]
}

Now if you want to move Todo Item 1 to Position 4, you can simply send this to your api:

{
  "todos": [
    {"id":1, "position": 4}
    {"id":2, "position": 1}
    {"id":3, "position": 2}
    {"id":4, "position": 3}
  ]
}

Answer 3

So, In not sure if I get what you're trying to do right, but if I understand this correctly, you have a Todo list reordering UI that is implemented by moving Todo items one position at a time, because maybe your UI used up/down arrow or something similar to reorder Todo items one item at a time, but you worry about race condition caused by your requests getting intertangled due to race condition caused by differing response times between each subsequent requests.

The simplest way to enforce this constraint is to use time-based conditional requests rather than ETag-based conditional requests. So instead of If-Match: <etag>, you'd send the current time in both the Date and If-Unmodified-Since in the request. The server should record the Date of last change against the Todo list and should conditionally perform the request only if the last updated time is older than the date in the If-Unmodified-Since header (Optionally, you can skip the Date header and just use If-Unmodified-Since for both).

That works fine if you only care about the side effect of the last reordering command, if your system somehow are made in such a way that the side effects that happens in the conditionally discarded requests are important, then you need to queue the requests either on the client or the server, depending on the consistency requirement that you're looking for.