Permissive vs. Strict API Message validation

Question

We had a interesting debate today over what our REST API should default to doing when validating a request body where unexpected fields are present. I think we ended the conversation in a good place, but it brings up somewhat of a meta point of whether or how to apply Postel's law that'd I'd like to explore further.

Our application is essentially a large and complicated order management, matching, and dispatching system (similar to a ridesharing service). Most of our endpoints are related to actions at various stages in the order input, matching, and management flows, in addition to entity management (CRUD). There's no plans to open this up this API publicly, so the only clients are our mobile and web apps (and potentially scripts we write).

I've written systems in the past where we where we took the liberal approach in accepting fields, where the situation was we were the receiver of an event stream that we had to handle. In that case the API must be liberal in accepting arbitrary fields, as otherwise the upstream producer can't effectively add features or change their system.

However, it seems to me in our case that this is a case where we'd want to be strict in what we accept. All of the users here will be internal developers, so it's hard to imagine many common situations where being permissive buys you reliability over a strict API. We've agreed that there are large improvements in developing vs. a strict API and many bugs that we can avoid by strictly validating during dev, however there are arguments for a permissive API being more reliable in prod by not bombing out on otherwise acceptable requests. However, since we control all the legitimate clients it seems like we shouldn't end up in this situation, except in certain rare cases of client releases that use some fields before the API is ready to consume them. So if we start seeing unexpected fields, we'd at least want to know and likely to error most of the time. Is there a common case I'm missing that where we'd be slowed down by a restrictive API.

There are some difference in background on the team, so some of this might be due to different frames of mind. I'd like to hear and understand other's opinions here as I don't fully buy arguments for permissive being more reliable and us wanting to serve clients who have unexpected fields.

Answer 1

Permissive vs. Strict REST API

It seems to me that your question isn't really about REST; similar concerns arise in other messaging systems as well.

I've found Greg Young's eBook to be a helpful overview of some of the concerns with changing message schema.

Is there a common case I'm missing that where we'd be slowed down by a restrictive API.

If your messages handling isn't forward/backwards compatible, then you are coupling the releases of your clients and servers.

Scenario: there's an important bug to fix in production on the server, and the new server code is ready; the client in production is fine, but the client in development has an important bug that needs to be fixed before you release. If the message handling is too strict, then you don't have the flexibility to manage a staggered release.

The hypermedia constraint in REST actually makes the schema problem easier to manage, because the server is providing to the client a form, which is a description of the schema that the server understands. If the client is clever enough to match fields on the form, use the default values provided by the server for unrecognized fields, and to ignore known fields that aren't provided on the form, then the server just needs to provide the representation that it understands.

(Example: Google search. You download the form, fill in the text field, and click the submit button without ever needing to consider the "hidden" fields provided by the server.)

It is useful to keep in mind Fielding's comment on the REST architectural constraints:

REST is intended for long-lived network-based applications that span multiple organizations. If you don’t see a need for the constraints, then don’t use them.

(Of course, if you relax the constraints, it isn't a REST API any more.)

Answer 2

OK so we are talking about messages that you could deserialise and use. But you are going to add extra validation and throw an error instead.

The situation should never happen, but it might occur when you upgrade APIs and make some deployment error. Some v2 api is getting v1 messages.

I guess the worry is that the v1 message is succesfully deserialised and processed, but the results of the processing will be incorrect. Say I change the numeric amount field from pounds to pence.

In such a case you would probably prefer an error over the incorrect processing and you can see how a simple verification achieves that without much cost in effort.

I suppose the arguement for permisiveness is really one of ease of manual client creation, rather than a preference for not throwing errors when a rollout goes wrong in a specific way.

If you are creating the client(s) for your api though then that arguement isnt really worth discussing.

Answer 3

The decisive factor in choosing between a strict or permissive interface is the expected use. What are the consequences of processing invalid input (or erroneously processing valid input) vs rejecting valid input?

Even ignoring additional unrecognized parameters could cause the software to produce "incorrect" results, e.g. if a later version can interpret the additional parameter, leading to another outcome.