Skip to main content
added 369 characters in body
Source Link
Flater
  • 56.3k
  • 8
  • 106
  • 157

Reinventing the versioning wheel

Does anyone know a standard approach/design pattern or method to this?

This sounds a lot like a versioning system. Are you sure you're not trying to reinvent the wheel?

Even if this isn't a case of reinventing the versioning wheel, it's still interesting to take some inspiration from versioning systems (Git, SVN, ...) on how the interaction between client and server works (you don't necessarily need to look at the specific implementation, the contract between them is relevant enough as-is).

It's no coincidence that I've used "push", "pull" and "check out" in this answer - all git commands. The similarities are quite accurate.


Top-level API design

As a sidenote, this is starting to sound like a versioning system. Even if this isn't a case of reinventing the versioning wheel, it's still interesting to take some inspiration from versioning systems on how the interaction between client and server works. It's no coincidence that I've used "push", "pull" and "check out" in this answer - the similarities are quite accurate.

  • The client specifically sends you a list of entity IDs that it has stored locally
  • The server already kept aits own record of which entities were checked out by which client and thus already knows this information

I suggest doing the former, as relying on the server's information can be an issue in case the client's local storage has changed without the server having been made aware of it. E.g. if the client's hard drive crashes and the application is reinstalled from scratch, then the server thinks that the client has local copies when in fact it does not.

Either way, you could significantly cut down on both query runtime and response data size by trimming the list to only contain entities that the client cares about.

The client now knows to simply deleteddelete these entities from its local storage. 

Note that if access can be revoked temporarily - then you wouldn't want to automatically delete the local changes without the user's consent, as they may want to keep these changes so they can push them when access has been restored.

Top-level API design

As a sidenote, this is starting to sound like a versioning system. Even if this isn't a case of reinventing the versioning wheel, it's still interesting to take some inspiration from versioning systems on how the interaction between client and server works. It's no coincidence that I've used "push", "pull" and "check out" in this answer - the similarities are quite accurate.

  • The client specifically sends you a list of entity IDs that it has stored locally
  • The server kept a record of which entities were checked out by which client and thus already knows this information

Either way, you could significantly cut down on both query runtime and response data size by trimming the list to only contain entities that the client cares about.

The client now knows to simply deleted these entities from its local storage. Note that if access can be revoked temporarily - then you wouldn't want to automatically delete the local changes without the user's consent, as they may want to keep these changes so they can push them when access has been restored.

Reinventing the versioning wheel

Does anyone know a standard approach/design pattern or method to this?

This sounds a lot like a versioning system. Are you sure you're not trying to reinvent the wheel?

Even if this isn't a case of reinventing the versioning wheel, it's still interesting to take some inspiration from versioning systems (Git, SVN, ...) on how the interaction between client and server works (you don't necessarily need to look at the specific implementation, the contract between them is relevant enough as-is).

It's no coincidence that I've used "push", "pull" and "check out" in this answer - all git commands. The similarities are quite accurate.


Top-level API design

  • The client specifically sends you a list of entity IDs that it has stored locally
  • The server already kept its own record of which entities were checked out by which client and thus already knows this information

I suggest doing the former, as relying on the server's information can be an issue in case the client's local storage has changed without the server having been made aware of it. E.g. if the client's hard drive crashes and the application is reinstalled from scratch, then the server thinks that the client has local copies when in fact it does not.

Either way, you could significantly cut down on both query runtime and response data size by trimming the list to only contain entities that the client cares about.

The client now knows to simply delete these entities from its local storage. 

Note that if access can be revoked temporarily - then you wouldn't want to automatically delete the local changes without the user's consent, as they may want to keep these changes so they can push them when access has been restored.

Source Link
Flater
  • 56.3k
  • 8
  • 106
  • 157

Top-level API design

You're trying to perform two jobs at once. It's beneficial to think of these as separate operations and analyze them separately.

  • Pushing the client-side changes to the server
  • Pulling server-side changes to the client

This means that both parties need to send information to each other.

  • If only the client gives information to the server, you miss out on server-side changes.
  • If only the server provides server-side changes, your client can't push its updates.

Now, you could stack these into a single web request (client pushes changes, receives server-side changes), but I would strongly advise against it as it's coupled too tightly.

So in conclusion, the better approach is to have two requests; one to push changes to the server, and one to sync changes from the server.

As a sidenote, this is starting to sound like a versioning system. Even if this isn't a case of reinventing the versioning wheel, it's still interesting to take some inspiration from versioning systems on how the interaction between client and server works. It's no coincidence that I've used "push", "pull" and "check out" in this answer - the similarities are quite accurate.


Fetching server-side changes

Currently, the focus is only on entities whose access has been revoked for the current user, so I'll only focus on that content.

A simple first approach to this is to have the client sent its lastSyncDate (i.e. when it last fetched server-side changes) to ensure your server doesn't send things the client already knows.
The server then looks for all revoked access since that date (which implies the server tracks timestamps of changes to access privileges), and returns the entities that were affected by these changes.

If this can lead to a massive list of entities and you really want to cut this list down to only contain entities that you know the client has copied locally, then there are two ways to cut this list down:

  • The client specifically sends you a list of entity IDs that it has stored locally
  • The server kept a record of which entities were checked out by which client and thus already knows this information

Either way, you could significantly cut down on both query runtime and response data size by trimming the list to only contain entities that the client cares about.

I don't know your underlying data store so which solution is the most applicable is not something I can decide.

The client now knows to simply deleted these entities from its local storage. Note that if access can be revoked temporarily - then you wouldn't want to automatically delete the local changes without the user's consent, as they may want to keep these changes so they can push them when access has been restored.


Pushing client-side changes

The only real issue here is how you deal with changes to entities on which access has been revoked since the last push. I can't answer that for you.

  • You could allow the changes if the checkout happened before the access revoking.
  • You could allow only the changes that were made before the access revoking, provided your local client stored (trustworthy) timestamps of all local changes.
  • You could outright refuse to accept any changes after the access has been revoked.

In all cases, this is a decision that needs to be made by the server. The client pushes its content (or tries to), and the server decides what to do with it. If the server rejects it, you return the error to the client.

While it is tempting to do so, you may want to hold off on having the client immediately remove the local entity when it encounters an error. Because you may in the future have other reasons for a push to fail (e.g. temporarily locked entity, database unavailable, ...) at which point you don't want your client to immediately remove its local changes.

I suggest for the client to rely on the fetch method to actually decide what to delete, because at that point the client has explicit confirmation from the server that access on these entities has in fact been revoked.