1

I have an API which allows me to communication with a device. The communication protocol is stored in a JSON file. It list the events that the device can raise, the functions, the frames format, etc.

But this JSON file changes often, and i need to keep my API backward compatible with all the JSON files.

I imagined some solutions (i am using Python):

  1. Use decorators on my functions (@api_1, @api_2 ...) but it will be too heavy when i will reach api 5 or more

  2. Make a folder for each API changes

    1. /APIs
      1. API_1
        1. frame_translation.py ...
      2. API_2
        1. frame_translation.py ...
  3. A strategy pattern ?

The second solution seems already better than the first one, but probably not the best.

How would you do it ?

2

In some cases, structures need to be shared across multiple systems running different program versions. Having multiple data structures to match each program version (as suggested above) is one solution but can also introduce multiple problems such as data cloning & synchronization, additional storage, added maintenance & complexity depending on your solution.

Another solution is to combine all data versions into one common structure and modify each program version to only process the information it understands and ignore the rest. The meme or versioning protocol is any data items within the data structure which is unknown is deemed to be data related to functions in higher releases and not needed for this program version (data ignored). Examples below..

All new version data must be appended to the end of a data structure. i.e. v1.0 only processes the first 10 bytes. v1.1 processes the first 20 bytes. v1.3 processes the first 40 bytes. This method ensures the first 10 bytes needed by program v1.0 is never altered so that v1.0 is programmed to parse the first 10 bytes and ignore the rest. If the 1st 10 bytes is an ID which v1.3 needs to be 20 bytes, not 10, the new 20 byte ID can be appended as example.

Implement General Markup language GML. The HTML protocol uses this method to serve the same web page to all HTML browser versions. Each browser is programmed to process the it has supporting functions for and ignore tags.

Another approach is to implement indexes in the shared structure to each versions data. This is similar to the above append method but using indexes of [offset,length] to locate the data version segment. All program versions understand the indexing and use it to locate the data it needs. to reduce data duplication, a data item is only located in the version segment which introduced it, i.e. a new program version does not need to change how a prior version field is obtained, specifically the code method in v1.0 which extracts a file_id is unchanged in code v1.4 and the file_id is still extracted from v1.0 data segment.

2

This is a variation on semantic versioning (which you should also read):

  1. Establish a version number for the API, with separate major and minor numbers. Include this explicitly in the JSON file and (if possible) in the protocol handshake. Start at version 1.0 (major.minor).
  2. When you make a backwards compatible API change, increment the minor number.
  3. When you make a backwards incompatible API change, increment the major number and reset the minor number to 0.
  4. When establishing a connection, if the major numbers don't match, fail the handshake or fall back to an older version of the code, if available.
  5. Don't make backwards-incompatible changes lightly (or at all).

I don't know the details of your API, but here are some examples of backwards compatible changes:

  • Introducing a new message type.*
  • Adding an optional field to an existing message, with the default behavior (when the field is not provided) identical to the previous behavior.
  • Weakening a precondition.
  • Strengthening a postcondition.
  • Marking a message type as deprecated, without removing or altering it.

Here are some examples of incompatible changes:

  • Removing a message type.
  • Removing a field from a message type.
  • Adding a mandatory field to a message type.
  • Changing the behavior of a message in a way that the other side of the API could reasonably notice (e.g. introducing a caching layer is fine, returning different results is not).
  • Strengthening a precondition.
  • Weakening a postcondition.

Technically, you don't really need the minor number at all. You could just rely on version control to keep track of this information. But version numbers are more user friendly than commit hashes, and you might want to do feature detection with the minor number.

If you never break backwards compatibility, you don't need the major number either. But it's a good idea to have it just in case you need to make a breaking change. Such changes should be rare.

On the other hand, if your API is not published yet, and you don't have any client code which might be affected by backwards incompatibility, you can start your major number at zero instead of one, to indicate the API is provisional. Then you can just make any backwards incompatible changes you like without touching the major number, and when you're ready to "go gold," you advance to version 1.0.

* If it's possible that the message recipient is older than the sender, you have to ensure that both the sender and the recipient are capable of handling the case where the recipient does not recognize a message, or a particular field of the message. In typical server-client web APIs, this is rarely a problem, but for hardware APIs the situation may be quite different. It's possible you would have to regard this as a breaking change, which makes some of this advice considerably harder to follow.

2

Given your current approach, the change in communication protocol shouldn't lead to API change: since the communication protocol is stored in a JSON file, the API structure remains the same for all the devices as soon as their communication protocol can effectively be described through JSON.

When communicating with a given device, the caller of your API simply has to specify the version of the communication protocol—either by specifying the JSON or by using an actual device ID and/or version, if it allows the API to find the corresponding JSON.

When you'll have to actually change the version of the API is if newer devices use features which aren't supported by the previous API, no matter how you modify a given JSON.

As a comparison, look at how USB standard works: you (luckily) don't have to update your operating system every time a new USB device is released; what you need, however, to do is to install the devices' drivers which indicate to the operating system how to communicate with a device—this is similar to your JSON files. However, USB 2 was superseded by USB 3 to provide higher speed and many other enhancements—something which wouldn't be possible by simply adding a driver.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.