Should a client ever test server response at runtime?

Question

I'm writing a client application that receives a JSON response from a server. In the past I've run into situations where a developer on the server side changes the JSON response in a way that causes the client application to crash. An example of this is when the client expects that a JSON field or subobject will always be present, but a change on the server side causes the JSON to deviate from what is expected possibly returning a null value when null should never be a possible response.

It seem like the server side could always have unit tests that ensure that the JSON response fulfills the contract, but that's susceptible to human error if a developer decides to rewrite a test or simply makes a mistake in testing or misunderstands a requirement. The client side can check that the JSON response is valid, but this would need to occur at runtime and if the server is writing proper tests, the double-checking of the server response by the client would be unnecessary.

Is there a recommended process to ensure that the contract (JSON response format) between the client and server doesn't get broken?

Answer 1

Yes, you should validate what you get, but you also need to be a Tolerant Reader:

http://martinfowler.com/bliki/TolerantReader.html

Martin Fowler states that:

My recommendation is to be as tolerant as possible when reading data from a service. If you're consuming an XML file, then only take the elements you need, ignore anything you don't. Furthermore make the minimum assumptions about the structure of the XML you're consuming.

Which, to my experience is a good way to be resilient (tolerant) to changes. Also, if the change is too big to be tolerant, you should consider some strategy to versioning your API:

1. https://blog.pivotal.io/labs/labs/api-versioning
2. http://www.mashery.com/blog/ultimate-solution-versioning-rest-apis-content-negotiation

From a more practical point of view, you can also write some test/contracts to your API so that you can continuously check the contracts:

1. https://github.com/realestate-com-au/pact (see the links at the end of page)
2. https://github.com/thoughtworks/pacto

Answer 2

Why on Earth wouldn't you? A good rule of thumb is to assume that all input is evil until proven otherwise. Blindly assuming that your input will be valid is a good way to wake up to empty databases and overflowing error logs. Just because that input is coming from a (presumably proficient) coworker's server doesn't mean it should get a free pass from you.

At the very least, you want a different error to be logged for Client and Server issues. If the owner of the server changes the response without notice, you want to be able to determine as quickly as possible the source of the error. Aside from covering yourself towards management, it helps to be able to tell at a glance the general area of a problem when your application stops working.