51

I was researching about best practices for standardised JSON response formats for APIs, according to various sources available online general consensus looks something like this:


//Successful request:
{
  "success": true,
  "data": {
    /* requested data */
  },
  "message": null
}

//For failed request:
{
  "success": false,
  "data": {
    /* error data */
  }

  "message": "Error: bad stuff"
}

My question is: what is the reasoning behind the "success" parameter inside the response body? Shouldn't the info about whether the request was successful or not be determined from HTTP status codes instead of additional parameters like "success"?

Also, many HTTP clients, like axios, will throw exceptions based on response status code, which can simplify the handling of requests. Example of using axios and status code exceptions instead of "success" parameter:


axios.get('/api/login')
  .then((response) => {
    // The request was successful do something
  }).catch(function (error) {
    if (error.response) {
      // Request made and server responded with HTTP status code out of 2xx range
      console.log(error.response.data);
      // Handle error json data in body
      console.log(error.response.status);
    } else if (error.request) {
      // The request was made but no response was received
      console.log(error.request);
    } else {
      // Something happened in setting up the request that triggered an Error
      console.log('Error', error.message);
    }

  });

I would appreciate it if someone could give me a few reasons why the standard with "success" param inside the json response is so common. There is probably something important I am missing related to motivation for such an approach.

8
  • 48
    Relying on HTTP status codes only works if every "link in the chain" is HTTP. The moment this payload gets saved to a DB, enqueued to a message queue, etc., then HTTP status codes no longer exist, so you end up need to fold them into your payload, anyway
    – Alexander
    Mar 23 at 1:45
  • 2
    @Alexander: I don't see why you would keep these. I found it a better design to generate them at one end of a pipeline and consume them at the other; if the response is large enough the whole thing is never materialized in RAM at once.
    – Joshua
    Mar 23 at 19:54
  • 4
    It's different layers, that's all. You wouldn't put the TCP handshake status in there either, right? Mar 24 at 12:23
  • 7
    Basically the HTTP status can be seen as the status of the "transport layer", while the JSON status response is from the "application layer" (note the double-quotes; no nit-picking, please!)
    – U. Windl
    Mar 24 at 12:25
  • 1
    @U.Windl I know what you mean - I think the names are technically incorrect but you get the point across. It's been a while for me too... Mar 24 at 12:27

7 Answers 7

56

A few potential reasons why you may wish to do this are:

  1. the fact that some HTTP clients treat anything other than 2xx as an "exception" to be thrown, which can hide differences between transport errors (like no connection, invalid firewall, invalid endpoint/URL, etc.) and actual API errors (bad API key, request validation rules, etc.), which could both end up being thrown as errors, which leads to extra work in determining which it was and extracting the actual API error text

  2. responses that aren't accurately / completely represented by normal status code, or even multi action requests, where you have >1 API action in a single HTTP request and each one may succeed or fail individually, in which case either a 2xx or 4xx would not be accurate

I personally prefer to inspect the JSON for errors instead of hoping that the language I'm using will easily differentiate between different kinds of errors. Java and C# for example would allow me to do multiple different catches of specific exceptions on a single try, but in JavaScript where anything can be an error and each try only allows a single catch, it's messier to separate transport errors from API errors

6
  • 2
    I think that you would have to determine if the error was transport related or api related anyway but you have the point when it comes to additional work for extracting the actual api error text since I don't think same error format could be used for all situations. About your second point, one could argue that 500 code was added exactly for that reason and you can return custom message for more info in error object. But you are right 500 in this case would be misleading if request was partially successful
    – Šime
    Mar 23 at 8:20
  • 2
    The caveat to the first point is that it makes handling certain errors more complex when you either have easy ways to catch specific HTTP status codes, or when the exact failure reason simply does not matter (or in cases where the devs can’t keep the payload consistent, like how GitHub’s API returns a 200 in some places for requests that were ratelimited, but uses a completely different schema for the error payload from what the request would normally provide). Mar 23 at 21:17
  • 3
    Or it's part of a layered system where only the payload is delivered, not the raw HTTP response.
    – OrangeDog
    Mar 24 at 13:24
  • the fact that some HTTP clients are terrible is not a good excuse to ignore a large part of the protocol being used
    – njzk2
    Mar 25 at 23:25
  • In javascript you would use promises, not exceptions.
    – Sulthan
    Mar 26 at 12:55
109

Many people take HTTP status code as “successful communication with the server”.

Now if a customer wants to buy a US$200 item and has only US$100 in their account, the JSON response will be “failure, insufficient funds”. But as far as HTTP is concerned, everything went just fine: It delivered a purchase order, and successfully returned back to the caller that the purchase failed.

So you get a status 200. You would get something in the 400 range if there was actually some communication failure. In that case you have no idea if there is money in the account or not. We didn’t get that far.

Note there are situations where we don’t even get an HTTP status: if the server is down, or if HTTPS negotiation fails, your application didn’t even manage to reach the server.

10
  • 28
    +1 For pointing out that not every involved party defined success and failure the same way, which is often overlooked.
    – Flater
    Mar 23 at 13:19
  • 12
    402 Payment Required literally exists. Mar 23 at 21:51
  • 46
    402 indicates a missing payment for accessing the API, not for the application served by the API.
    – chepner
    Mar 23 at 22:24
  • 6
    @imel96 "In fact, you don't get HTTP response for communication failures" - except when you do, e.g. when services are running behind reverse-proxies and report their upstream network/communication failures as "normal" HTTP responses to service clients - or even when it's a normal proxy operating in your own local network (which is less common nowadays, but was very popular in the 1990s).
    – Dai
    Mar 24 at 7:28
  • 18
    Many people take http status code as “successful communication with the server”. Then many people are wrong about this. The 2xx, 3xx, 4xx, 5xx status code categories have very clear semantic meanings far beyond whether the communication with the server was successful or not. In fact, almost all are successful communications with the server, except for a handful which may be generated by a proxy. If the server tells you that he encountered an internal server error, that was still a successful communication...
    – AnoE
    Mar 25 at 9:56
18

There are inherent shortcomings in trying to fit a nuanced, complete API into the limitations of HTTP. The above examples provide some good points to why that's the case.

Here's another scenario we keep running into where I work. If you get a 404 back, does that mean the endpoint doesn't exist or that the query by ID for an item didn't find a corresponding item to return?

You CAN return a 204 in the latter case but that can be taken to be ambiguous as well depending on the situation. Providing more detail in a JSON response seems the way to go. It allows you to precisely distinguish between all those cases.

We've negotiated that 404 only be returned for endpoint not found with JSON data being returned for the other cases along with a 2XX return code.

6

The other answers are good and cover most of the reasons I know of for this pattern. I'll add one more from experience: in some cases the request may be indirect (i.e. proxied from the initial endpoint to some other internal API), and only the response body can be reliably propagated through the chain.

This is most likely in a legacy situation, e.g. where response codes have been used inconsistently in the development of several different services, but the front-end needs to use all those services and deliver a sane UX.

1
  • 5
    Opposite but related: I don't know if it's still like this, but at least at one time apache would completely remove the response body if it got specific error codes from the backend. So when our json responses relied on http status codes, it worked totally fine in development with django runserver, but was broken in production behind apache.
    – Izkata
    Mar 23 at 20:06
1

Short answer: Because the overall success of the request does not only depend on the technical success of the HTTP-request but also on the business-logic validation success of the data.

3
  • 4
    The issue probably is that the HTTP status code does have semantic meaning beyond the technical success of the HTTP request. I.e. "access denied" or "no data found" or "created" or "accepted" or ...
    – AnoE
    Mar 25 at 9:59
  • @AnoE I include those meanings in "technical". By validation I mean simply business-logic validation. E.g. "Username may not exceed 128 characters -> success=false" which is not a technical constraint. Mar 25 at 10:08
  • 2
    Sure, some validations cannot be expressed in HTTP codes; but others can. It is unclear to OP why one needs an additional JSON body for "no data found". Generally, these short answers are downvoted because people think that some aspects are missing from it, I'd suggest to add whatever you write in the comments to the answer, and it will make it more full-fledged.
    – AnoE
    Mar 25 at 10:47
0

On the one hand it is perfectly fine to rely only on the HTTP status codes for what they are. If your API is simple enough or restricted to enough consumers - for example, an app that is developed by a single, smallish team, with an Angular frontend and some Python backend. In that case, skipping the formal body, and just using HTTP codes for the vast majority of cases is perfectly fine.

In that case, obviously if you need to transport some data back to the client in response to the request, then you'll use a body as usual, but that is then just an implementation detail of that specific request. There is no particular reason to enrich every single request with a JSON body.

This is especially the case if you have many small calls that are in themselves relatively simple, and where you expect them to work 99% of the time except for actual errors - TCP/IP problems, DB issues, session timeouts, or straightforward actual bugs in the client or server. All of these cases are easily covered by standard HTTP codes, and often the client would not really care about the exact reason anyways, but it's enough to know if it's a 2xx, 3xx, 4xx or 5xx.

Now, about the other cases. What reason would there be to add additional error information in the result of an API call. Possible cases:

  • If you are providing an actual API to some external consumers; i.e. you are not developing the consumer right along the API yourself. Then you need to give some additional info beyond the code. Obviously you still have to think a little bit about what info you give; i.e. you do not want to make it easy for an attacker to figure out what went wrong. In this case you might as well define a common format for the JSON body and use it all the time. That said, you could still opt for a shortcut and define your "standard" response such that if there is nothing interesting to say about the response, then an empty body has to be accepted by the client as well, if you prefer so. Matter of taste and opinion.
  • If you need to report actual hints to the user - for example "No search results found" vs. "Too many search results found, change your criteria", then that message (or a code which is then transformed into a message by the client) indeed needs to go into the JSON body. In this case, you as the API designer have to decide whether you want to have a standard body as a kind of additional "transport layer" for all API responses, or whether this message just goes into a response body for that particular request. You'd use the proper HTTP code in addition, anyways.
  • If you wish to easily be able to switch from HTTP to other schemes (i.e., pub/sub schemes, queues etc.), then you simply have no HTTP status code (and nothing else from the HTTP header, including the URL, URL parameters, HTTP method etc.) and need to encapsulate everything in the request body and response body. One can argue that this is not a "real" reason though, in some sense. A HTTP message does have other semantics than something pushed to a queue or into a DB table, and I can think of plenty of cases where a perfectly fine HTTP communication would make zero sense if moved into a different context, in the first place.
  • Finally, a reason certainly not to be ignored is that sometimes there are simply architectural restrictions which are dumped on you. If you are working for a client who has simply decided that this is the way it should be in their software, then there you go. It wouldn't be the first time, that something seems useless unless you look at the big picture - having a common format over many, many microservices, APIs, etc., and relieving the individual developers from the chore of deciding these things themselves over and over again, and making it easier for devs to switch between teams and finding the same conventions over and over again, may be a worthy goal in itself. And this usually leads to more boiler-plate like the one you are describing.
0

I asked about this seven years ago: Do web applications use HTTP as a transport layer, or do they count as an integral part of the HTTP server?, and it still hasn't been answered in a satisfactory way, so I chose my own direction.

Point being that I now justify wrapping every response like this:

{
    "success": true,
    "status": 8005,
    "data": data-or-null
}

I have two reasons for this:

First, various intermediaries could render a response instead of the API. Be it the CDN/load balancer/reverse proxy/webserver running in front of my code. I don't necessarily control that server, so maybe it'll start spewing HTML over 500 Internal Server Error pages when something is amiss. When my clients don't get a JSON body with "success": true, they can safely assume they can't handle the response.

Second: HTTP status codes are limited, and discussed more endlessly than the orientation of the toilet paper roll or on which side of the football the laces are supposed to be. Just responding 200 (okayish requests), 400 (that's not JSON), 422 (the user sends something I can read, but definitely can't process) and 404 (user requests a single resource, but that does not exist) covers all scenarios that my application and client can handle.

Having your client expect a certain body allows you to better verify that it can handle it. Using your own status codes within the body allows you to convey more different and specific error scenarios, which the client can then translate into a machine- or human-readable status message.

4
  • that one is easy: footballs don't have laces
    – njzk2
    Mar 25 at 23:29
  • Your linked question has REST tag, so I'd say in many cases, returning "success": true is a sign doing RPC rather than REST. There are enough status codes to say about the object representation sent to the server. It doesn't make sense to say "success" when receiving an object. E.g. if client sent an "inflated football", the server can say "Ok" (REST). If a client say "inflate the football" then the server can respond with "success" (RPC).
    – imel96
    Mar 30 at 6:00
  • @imel96 most APIs aren't pure REST, not everything is treatable as a resource.
    – CodeCaster
    Apr 2 at 11:46
  • @CodeCaster yes, maybe. What I'm saying is HTTP status code is sufficient for REST. HTTP status code is not needed for RPC. It's up to the developer what to use if they mix REST and RPC together
    – imel96
    Apr 4 at 0:23

Not the answer you're looking for? Browse other questions tagged or ask your own question.