Serialization pattern and code duplication

Question

I noticed how our codebase contains multiple versions of the same method, which unmarshalls the inbound byte-stream into java objects and that the only difference between the current and previous version is the one variable change. There are roughly 17 versions, which are all chosen based on one big switch case. Similar differences occur between any version, where a parameter has changed, and it has caused the whole method to be duplicated.

Now the way I understand it is that parameters read from the byte stream come in order as they were placed there and any unmarshaller would need to be aware of the order in which the elements were added to the byte stream. Any changes to the order of the byte stream and a whole method is rewritten and new version added. This leads to significant code duplication as majority of the parameters within the method remain unchanged.

My question is whether this is the standard way of implementing this or is there some pattern/design principle which could be followed which would not lead to such duplication for reading elements from the byte stream. From architecture perspective, the system is a big monolith.

Answer 1

My recommendation would be to use a serialization format/library that allow for minor changes, like adding or removing properties. So you would create your class and ask the library to serialize or deserialize an object. In some languages you can use type annotations to instruct the library exactly how serialization should be done. Any added properties would need a default value, and any removed properties will simply be ignored. This is typically how json works.

It is also common to use separate classes just for serialization, sometimes called "Data Transfer Objects" (DTOs). If you make larger structural changes it can give you more flexibility to keep the old class to handle legacy data, while saving data using the new class. This idea also help with separating serialization concerns from the logic of the class.

If you are serializing/deserializing the byte stream by hand it is probably a good idea to reuse existing code if you can. So if you add some properties your DeserializeV2 will first call DeserializeV1, and then read any extra properties from the stream. But this would not be possible if you are removing properties, so duplication might still be required in some cases.

And duplication is not necessarily a problem. After all, each version fundamentally represents different things, how the object was stored at a specific point in time. The usual concern with duplication is that you need to update multiple places if there is an issue, and that is probably not relevant since a specific version should never change.

I would also recommend using some form of automated tests to ensure legacy data can always be deserialized. This will likely be cumbersome to do by hand, so automated tests can really help ensure reliability.

Answer 2

Yes its common to duplicate de/serialisation classes.

The reason is that you often have multiple versions of the serialised data lying around in different places concurrently. So you can't just replace V1 with V2 a serialiser, as you still need the V1 deserialisation code (preferably unchanged, no new bugs) to read your data!

I think what's uncommon is keeping 17 old versions in the codebase. Although I can see possible reasons, I think its more usual to run a clean up on all your data to bring it to the latest version and remove the older serialisers. Perhaps this is seen as an unneeded risk?