How to handle mapping from object model to different versions of contracts or DTO's

Question

I'm looking for patterns or best practices for maintaining different versions of an API contract. There's lots of information on the internet about how to version API's (URL's vs. headers etc), but this question is related to the mapping from one object model to different versions of the contract types (DTO's), and for that I haven't been able to find any information really.

Imagine you have a web project with an endpoint, and a client app (Could be desktop or mobile) that communicates with the web project using that endpoint. The endpoint is only used by your client app, i.e. it's an internal endpoint, not public. The communication between the web project and the client app works by the client app sending some data to the endpoint, and the web project receives it, processes it and sends back a response object. There's a contract for how the structure of the data sent to the endpoint and sent back from the endpoint should look like. It's fairly complex in terms types of the number of object types and their nesting. You can't control when your users upgrade their client app, and it's a business requirement to be backwards compatible.

You have an object model that evolves and that's no problem since you've separated the API contract (The DTO's) and the object model. You have mapping code that maps from the object model to the contract and vice versa.

You can make non-breaking changes to the contract, but if you need to make breaking changes, you'll need a new version of the contract, and you'll need to make sure you can map from your object model to the new contract. But as you need to be backwards compatible, you also need to make sure you can somehow map from the object model to the previous version of the contract.

My question is then: How would you architecturally handle this? Do you know of any patterns or best practices that can be used for this?

The challenge is when you change the object model and create a new contract, you also need to make sure that there is some way that the old contract keeps working. One way I see this could be done, is by maintaining a mapping from the object model to each version of the contract.

Another way could be to only maintain a mapping from the object model to the latest contract, but then maintain a migration path from the different versions of the contract (So when needing to map from v3 to v1, you would map from object model to latest contract (v3), and then map to v2, and then map to v1).

Another issue to consider is the amount of duplication. When the contract has a lot of types, and you need to just delete a single field from one of the types, this is a breaking change and requires a new version. Would you copy the whole contract (All types) to a new folder in your project, and then delete the one field? This would mean you have an almost identical contract, and a lot of types would be duplicated. Duplication in this instance can be justified, but is there a better way?

Any pointers, insights and suggestions are appreciated.

Answer 1

The problems you describe exist in identical form in object serialization systems, and these problems have all been worked out over the years. What you describe as a contract can also be viewed as an object serialization format, at a higher level.

For serialization, generally speaking, you have only a single version of the "contract" implemented. This implementation supports the latest version with no conditional paths taken; i.e. the default, most straightforward code path supports the latest version. To maintain backward compatibility with previous version, conditionals must be introduced based on the type of object change:

• Removing a property (I use the word property as in .net, but it is any public field in an object): Normally you do not need to do anything for backward compatibility.
• Adding a property: If you provide a useful default value for the property, the API user does not need to initialize the new property. I use the word "property" to also refer to data that is orthogonal: one property does not affect another. This is part of good API design and a requirement for backward compatibility.
• Change the type or name of a property: in this case you have to add conditional code to examine the version of the API consumer, and convert the name or type to the new name or type.
• Change the name of a data type: the change is similar; in some cases, you must keep the old data type around so that it can be loaded with data before conversion.

Now consider API compatibility instead of serialization compatibility. In many ways, this is a more difficult problem, because you must provide the entire API in the exact version needed. While you can add to that API, you cannot change the functionality of the old API in any way. You must keep all old data types and all old properties. The general strategy is this:

• Add new functionality and new data types without interfering with the old API. Adding new properties is no problem.
• Users writing new code will use the newest version.
• Users working with previously written code have no problem as you support the entire old API.
• Keep track of the versioning age: At some point, it becomes too difficult to maintain the old version. At this point, you create a version that has all the old compatibility code stripped out. People using older versions of your API can still link to older libraries.

The main strategy with API evolution is never change it. This means your API must be very well designed, with thought given to making the most general data types, not the most specific. You must plan ahead for years. It helps to have years of experience in the specific domain. Even with careful planning, the API must change, but at least you can minimize the changes.