Is building REST API compatible with Agile approach?

Question

I believe I understand what REST is and what Agile is. Our company has some APIs that we maintained, usually we only needed to tweak some requests or responses to fit new requirements. Recently, we had a need to create new set of API.

The design of the new API was driven by how the client would interact with the server. Because the interactions are simple and had been agreed between the client and server teams, the API will only support a couple of requests, i.e. PUT & GET requests for purchase and a POST request for cancellation.

It then become obvious that the API is not REST at all (although both teams agreed to make REST API). For example, there's no link relation anywhere to drive the client and there not even an initial URI that can drive clients.

One example that prevented to implement something closer to REST is that because the client already how to interact, then adding link relations to responses would add no value to the user, i.e. not agile.

REST is software design on the scale of decades: every detail is intended to promote software longevity and independent evolution. Many of the constraints are directly opposed to short-term efficiency. Unfortunately, people are fairly good at short-term design, and usually awful at long-term design. Most don’t think they need to design past the current release.

That's a quote from Fielding, which I also find hard to reconcile with agile. We don't expect our API to still be used after a decade. If the API passed the tests, we'll release it and that should be the end of it, exactly what he said.

So, the question is, is it true that we can't make REST API in agile development and that you have to choose between REST or agile? Any example of real REST API that was done with agile process?

By REST here I mean an API where you can use it just how people browse the web, without relying on documentation pages.

---

To clarify why this is different from doing architectural design in with Agile approach, there was no real design work other than to agree that it would be a REST API. We pretty much went straight to implement the elements that's really needed and the end result (which met the user requirements) is not really a REST API.

Answer 1

So, the question is, is it true that we can't make REST API in agile development and that you have to choose between REST or agile?

No, not at all. Agile is just a way of building software in short iterations. REST systems are designed to last a long time, but that is nothing to do with how long they take to build. You can build a REST system in a few days depending on how complicated the resources and states they can be in are. And you can build them in an agile development workflow no problem

You are documenting your API end points. In REST you wouldn't bother doing that, you would instead document your content types and the states they can be in. When you have that your API is pretty much already documented since HTTP 1.1 defines how to get your various resources into different states.

In many cases this actually makes it easier to work in an agile work flow since you are only nailing down what is unlikely to change (the actual definition of the resources and their states) and things that could change quite a bit (the URI of the resources, the URI schema of your API) can change without you having to change the documentation of your API.

Anyone who has had to implement nonsense like this myapi.com/v2/some_resource, myapi.com/v3/some_resource, myapi.com/v5/some_resource because they have told the client about a particular URI schema and then changed it due some tiny change can relate to the pain of that. REST doesn't have that issue.

Of course if this isn't how your client thinks about things and you want to build something for what your client is familiar with, then it is perfectly valid not to do this. Obviously teaching your clients what REST is and how to think about it will add time to any project if they aren't familiar with the concepts.

But that would be true with anything, and is not an issue with agile

For example if your client uses Ruby you will probably write a Ruby client for them instead of a Go client, even if you think a Go client is better suited. It is common for developers to make design decisions based on what clients will be familiar with.

But this really has nothing to do with REST or Agile.

Answer 2

Agile organisation and API design are not opposed. They can interact in some way, but these two activities focus on different, mostly unrelated things.

Many REST APIs in the wild stop before level 3 in the Richarson maturity model (HATEOAS). It may be a bad thing in some cases, but not always, as long as one does not limit their view of an IT project through an exclusively technical lens.

Can you evaluate the value it provides, and to whom, in your current project? Just from your description, it is for me hard to say what HATEOAS would provide in your case. So from an agile perspective, it is less of a priority than other tasks in your project. But do note that "Agile" is not at odds with that kind of API design. You have projects where agile teams implement that kind of APIs. Because they have needs and constraints that make HATEOAS a valuable choice.

Also POs are not omniscient (though a good PO does not do too many mistakes). It may be that HATEOAS is useful in your case. If that's the case, agile practices are likely to make that clear sooner rather than later through their feedback loops.

Answer 3

I'm going to disagree with @KevinLH (sort of)

Yes technically agile is a project management methodology and REST is a technical spec. There can be no incompatibility between them because they are in different fields.

Unless!.... we accept that some REST features ie HATEOAS are always 0 or negative value and should never be implemented.

Given that many REST features are always 0 or negative value I would agree that yes there is a conflict between Agile, which in general says "Don't do useless things!" and REST which says "Do do useless things!"

But I'm sure we would all agree that the question there is really "Is REST good?" rather than does it conflict with anything.

Answer 4

Because the interactions are simple and has been agreed between the client and server teams

I think this last is an overlooked element here. In 2008, Fielding wrote:

REST is intended for long-lived network-based applications that span multiple organizations. If you don’t see a need for the constraints, then don’t use them.

(Emphasis added).

To my way of thinking, the conflict here isn't between REST and Agile, but rather the fact that your "customer" doesn't benefit from having a hypermedia API built upon a foundation of readily standardizable forms.

But in conditions where the customer would benefit? then I don't think Agile practices will get in the way of delivering a REST API at all.

Answer 5

Agile is about doing the things which provide the most value to the customer or product owner. There is nothing special about REST in this, it applies to any feature or task you might spend effort on: Is this the best possible use of your time, of would you generate more value by working on something else?

You have an API which fulfills its purpose in its current form, but it is not really REST compliant. The only reason to extend effort to make it REST compliant seem to be that you at one point decided that it was supposed to be REST. That line of thinking is definitely not agile.

REST can of course be perfectly fine in an agile project if you actually need it and it provides value. But it seems you don't.

Answer 6

This is a common challenge when we get down to the details of software development in Agile. We run into the same apparent conflict in the Open/Close principle of OOP or when we try to apply architectural patterns. Here's the thing we need to remember:

Anyone who has real-world experience and has written on development principles that look long-term knows that in reality, needs change constantly and the perfect architecture is effective only as an ideal to strive toward.

Anyone who has real-world experience and has written on Agile knows that techniques promoting perfect adaptability are an ideal to strive toward, but can lead to sub-optimization and getting trapped in bad decisions, and so must be balanced with long-term context.

As professional developers, part of our job is to find the balance between the two that gets many of the benefits of each while avoiding most of the pitfalls of both.