Planning Dynamic Patient Charts with Version Control and Historical Accuracy - Temporal Versioning or JSON

Question

I'm working on a project for our healthcare clinic's software and could use some guidance. For regulatory and historical purposes, patient charts need to remain as "static" records—representing exactly what the provider saw and entered during a specific visit. However, in our system, patient charts are dynamic and consist of a series of forms tied to a parent note type.

For example, if a patient is seen for pain, there could be several forms included in the note type, such as an HPI, pain therapy forms, and pain assessments. Administrators can update these notes and forms, including the controls within the forms, to improve charting workflows or meet new requirements. This flexibility is useful but complicates maintaining historical integrity.

When notes and forms are updated, I need to ensure that providers can view previous visits exactly as they were documented, even if the structure of the notes and forms has since changed.

Example:

• Visit 1: Provider A uses Note.v1 with Form A.v1.

• Later, an administrator updates the note to Note.v2 and the form to Form A.v2.

• Visit 2: Provider A now uses the updated Note.v2.

• However, when revisiting Visit 1, Provider A should see the original Note.v1 and Form A.v1, along with the historical data values from that visit.

Currently, this functionality exists in our system but was poorly implemented by the previous developers. They stored large chunks of HTML in string fields in the relational database and injected them as views. This made maintenance and data synchronization a nightmare, not to mention introducing risks due to improper handling of data.

I want to move away from that approach by rendering forms server-side based on a structured data model, abstracting away raw HTML. This would simplify maintenance, allow better state management, and avoid the need to parse the DOM to retrieve data (yes, that's what they did). While the original approach accidentally served as a snapshot mechanism for form versioning, it’s not scalable or maintainable.

I'm torn between two approaches:

1. Versioning Schema for Each Mutable Item

   • Create a versioning schema where every change to a note or form generates a new version and archives the previous one. Visits would be tied to specific versions of notes and forms, allowing the system to reconstruct views from the data model during the relevant time period.

   • Pros:

     • Robust and enterprise-friendly.

     • Accurate point-in-time reconstructions.

   • Cons:

     • Requires manual synchronization across multiple tables.

     • Complex to implement given my current tech stack (ASP.NET 3.1, EF Core 3.1.4, SQL Server 2017).

     • Temporal tables aren't an option without major refactoring.

2. JSON Snapshots for Each Visit

   • Capture a JSON snapshot of the form structure at the start of each new visit and store it with the visit record. When accessing a historical visit, the system would reconstruct the view using the JSON configuration.

   • Pros:

     • Simple to implement.

     • Efficient for real-world scenarios, as JSON deserialization is lightweight for this use case.

   • Cons:

     • Slightly redundant storage.

I'm leaning towards option 2 for its simplicity (I'm a solo engineer). I would have loved to use modern tooling and all its conveniences but updating to an LTS framework isn't in the works right now.

Answer 1

I think you are likely to have a much better result with the JSON document option or something similar. The main reason being this:

When notes and forms are updated, I need to ensure that providers can view previous visits exactly as they were documented, even if the structure of the notes and forms has since changed.

That requirement is going to be extremely difficult to support using a versioned database schema. It's possible/likely the only realistic approach to do this successfully will be to actually have different tables for different versions of the forms.

I wouldn't worry too much about the storage. JSON is a little bulky, but it tends to compress really well.

Also, if you are not familiar with FIHR, you should at least look it over. Specifically, look at the clinical and diagnostics modules. There may be structures defined there that meet or are close to meeting your needs. A lot of regulatory requirements are expected to require or allow FHIR formats anyway. Check the implementation guides for relevant specs.

Answer 2

You have one complete set of data, split into partial sets. Each set can have different versions (in the future).

As you describe, the patient chart is created once and never, ever changed. However, a patient chart created next week could contain one partial set of data created with a different version.

Say you have one set of data for the patient's identity. Whenever such a set is created it is created with the current version number and corresponding data. One set contains everything about the heart rate. Again, you can have different version numbers.

I assume that the patient chart could in principle contain any set of version numbers. Then you need code for patient identity, heartrate etc that can each handle data with any version number. Partial data that must never be modified would be marked as "unmodifiable". If you have 10 patient charts for a patient, each can use different version numbers, but each ones data is marked as "unmodifiable".

JSON structs are quite useful for storing data that could come in different formats. Yes, storing data as html is a serious offence. Ask your lawyers if you are at least allowed to modify existing data without changing its meaning.

Answer 3

The simpler, the better!

For capturing historic snapshots which will never be changed afterwards, redundancy is totally irrelevant (or even a requirement!). Storage is cheap today, so spending a few kb more per snapshot is unlikely to become an issue. In fact, you may even consider to save the snapshot as PDF or PDF/A (or maybe dome image format), not as a JSON where your software still needs to reconstruct the exact layout (and may miss that mark because of some bug in the past). PDFs or standard image formats have the advantage they are likely to be readable in a few decades from now. Are you sure your "JSON form reconstructor" will still work that at time?

But if PDF snapshots do not work for some reasons, go with the most simple, most robust method you can come up with. If that is saving a JSON form, use that.