3

I have a JSON object of arbitrary complexity, representing several entities (not my design choice). I want to partially persist it, only for the changed entities. What would be the best way to mark the parts of it that I will update? (Think "dirty bit"). The JSON will be passed around several microservices (again, not my choice), and it will only be persisted to the DB at the very end.

Possible implementations:

  • Generate a dirty flag for each subobject, which is tedious and error-prone.
  • Keep a separate, sparse object that represents a "patch", and iterate this "patch".
  • Keep a copy of the original, and change the pass-around object. Finally, run a diff of the two (slow) and persist the changed sub-entities.

Is there any other way that makes sense and I should consider?

  • 1
    JSON is only a format, regardless of where and how it is stored. So your interesting question needs some context. Since you want to partially persist it, I understand that it’s an object in memory. What is your object ? what language ? why persist it partly and not in whole ? – Christophe Oct 21 '19 at 7:46
  • Christophe, it will be a microservice architecture so there is no set language. Usually, it will be Kotlin/Java. – JoseHdez_2 Oct 21 '19 at 7:52
  • 2
    I think this is an XY question. You need to provide a larger context of what you are trying to accomplish. – JimmyJames Oct 21 '19 at 13:55
  • 1
    It's not clear why you need to 'mark' anything. Can you at least explain what that's about? – JimmyJames Oct 21 '19 at 15:22
  • 1
    A concise example might make your question more clear. – Dan Wilson Oct 21 '19 at 16:06
5

Sounds like you want to use the JSON Merge Patch algorithm. From the abstract:

This specification defines the JSON merge patch format and processing rules. The merge patch format is primarily intended for use with the HTTP PATCH method as a means of describing a set of modifications to a target resource's content.

| improve this answer | |
  • I wanted to be sure whether this would be the "best" way to process changes in a big JSON. An RFC implementation to follow is a big plus. Thanks! – JoseHdez_2 Oct 22 '19 at 7:55
3

One possibility is computing a hash of the data and storing that as well. When if comes time to save, compute the hash of the incoming data and compare it with the hash that is stored for that object. If it is different update/insert it. If the same, do nothing.

This will add some processing time. As long as your data isn't huge, it should not be that noticeable.

| improve this answer | |
  • 1
    Good idea, this is essentially how Git works. – Graham Oct 21 '19 at 20:20
  • 1
    Furthermore/more specifically, by storying the hash of sub-objects, you can find exactly what changed within a JSON blob. – Phoenix Oct 22 '19 at 2:39
1

If reading and writing the complete JSON object is fast enough, do that. If it is not fast enough, don’t use JSON.

There’s nothing that stops you from saving parts of your data as individual JSON objects.

| improve this answer | |
  • I wish it were that simple, but the JSON object is an amalgam of several different entities (not by my decision) that should not be persisted again every time. – JoseHdez_2 Oct 21 '19 at 8:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.