77

When I first started learning PHP (about 5 or 6 years ago) I learned about Ajax, and I went through "the phases":

  1. Your server returns HTML data and you put it inside a DOM's innerHTML
  2. You learn about data transfer formats such as XML (and say "oooh so THAT'S what it's used for) and then JSON.
  3. You return JSON and build your UI using vanilla JavaScript code
  4. You move to jQuery
  5. You learn about APIs, headers, HTTP status codes, REST, CORS and Bootstrap
  6. You learn SPA, and frontend frameworks (React, Vue.js, and AngularJS) and the JSON API standard.
  7. You receive some enterprise legacy code and upon inspecting it, find that they do what's described in step 1.

As I worked with this legacy codebase, I didn't even consider that it could return HTML (I mean, we're professionals now, right?), so I had a hard time looking for the JSON endpoint that was returning the data that the Ajax calls populate. It was not until I asked "the programmer" that he told me it was returning HTML and being appended directly to the DOM with innerHTML.

Of course, this was hard to accept. I started thinking of ways to refactor this into JSON endpoints, thinking about unit testing the endpoints and so on. However, this codebase has no tests. Not a single one. And it's over 200k lines. Of course one of my tasks includes proposing approaches for testing the whole thing, but at the moment we're not tackling that yet.

So I'm nowhere, in a corner, wondering: if we have no tests whatsoever, so we have no particular reason to create this JSON endpoint (since it's not "reusable": it literally returns data that only fits on that part of the application, but I think this was already implied since it... returns HTML data).

What exactly is wrong with doing this?

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

  • 2
  • 3
    Also related: stackoverflow.com/questions/1284381/… <- A very good answer in SO. – Machado Mar 14 '17 at 17:22
  • 72
    A server using HyperText Transfer Protocol returning HyperText?! The horror! – Andy Mar 14 '17 at 23:01
  • 3
    @Andy To be perfectly honest, it's actually Generic Message Transfer Protocol: nothing about it is specific to transfering hypertext, opposed to FTP, which talks a lot about things specific to files and directories. – Joker_vD Mar 15 '17 at 9:23
  • 4
    @Joker_vD I've never heard of a protocol called GMTP. While things have evolved and you can send other types of content over HTTP, it wasn't the original intent. My point is that just because you can send other content besides HyperText using HTTP, it seems silly to suggest that its not valid anymore to use it for its original purpose. – Andy Mar 15 '17 at 21:07

12 Answers 12

115

What's actually wrong with an endpoint returning HTML rather than JSON data?

Nothing, really. Each application has different requirements, and it may be that your application wasn't designed to be a SPA.

It may be that these beautiful frameworks that you cited (Angular, Vue, React, etc...) weren't available at development time, or weren't "approved" enterprisey thingies to be used in your organization.

I'm gonna tell you this: an endpoint that returns HTML reduces your dependency on JavaScript libraries and reduces the load on the user's browser since it won't need to interpret/execute JS code to create DOM objects - the HTML is already there, it's just a matter of parsing the elements and rendering them. Of course, this means we're talking about a reasonable amount of data. 10 megabytes of HTML data isn't reasonable.

But since there's nothing wrong with returning HTML, what you are losing by not using JSON/XML is basically the possibility of using your endpoint as an API. And here lies the biggest question: does it really need to be an API?

Related: Is it OK to return HTML from a JSON API?

  • 4
    I'd take a step back before saying it's just "simply preference". There are some "big deal" decisions you have to make: Is it an API or not, do I have the proper libraries to work with this as JSON data on the client, which type of client will I support (browsers without Javascript, for example), what volume vs. CPU time I have available to use, which strategy my programmers will leverage better, etc. etc. etc. – Machado Mar 14 '17 at 20:36
  • 7
    “it won't need to interpret JS code to create DOM objects - the DOM objects are already there, it's just a matter of rendering them” — well, the HTML is already there (once it’s arrived over the wire). The browser has to parse the HTML and make the DOM objects from it. – Paul D. Waite Mar 14 '17 at 23:33
  • 7
    There's zero reason that HTML can't act as an API. Zero. None. In fact, HAL+JSON and HAL+XML bear a striking resemblance to HTML. Here's an excellent talk about REST. Relevant part about returning HTML from an endpoint is near the end. youtu.be/pspy1H6A3FM Personally, I make all my endpoints return both json and HTML. If you're writing discoverable services, it makes it really easy to browse it in... gasp... a browser. – RubberDuck Mar 15 '17 at 10:36
  • 4
    Because it's a complete bitch to extract the data you actually care about to try to use it in any new way? – DeadMG Mar 15 '17 at 11:12
  • 4
    Serving HTML over HTTP? What is this? A website? – Ant P Mar 16 '17 at 21:07
50

JSON and HTML fulfill two different semantic purposes.

If you are populating a web page with data, use JSON. If you are building up a web page from portions of web pages, use HTML.

They may kinda sound like they're the same thing, but they aren't, at all. For one thing, when you are building up a portion of a web page using HTML returned by the server, you are working server-side. When you are binding data to a web page, you are working client-side.

Also, you have to be careful with HTML to not tightly-bind to a specific page. The whole point of rendering partial pages in this way is for the partials to be reusable, and if you make the partial too specific, it won't compose to other pages. JSON doesn't have this problem, because it's just data, not web page structure.

  • 1
    "For one thing, when you are building up a portion of a web page using HTML, you are working server-side." Why? I see no reason why this should be the case. It's only obviously true for the initial page load and arguably, not even then since the client can make requests for the data it needs. – DeadMG Mar 14 '17 at 21:34
  • 3
    @DeadMG It should say "when you are building up a portion of a web page using HTML returned by the server" (as opposed to using JSON returned by the server) – immibis Mar 15 '17 at 2:01
  • Indeed, but since there is little motivation for that to ever be the case, I don't see the point. – DeadMG Mar 15 '17 at 11:11
  • 6
    @DeadMG Little motivation for what to ever be the case? For the server to return HTML? That is literally what this whole SE question is about. – immibis Mar 15 '17 at 23:20
  • The question is about whether or not you should return HTML. It's clear that the initial response must be HTML but there's no obvious reason why any other APIs should return HTML. – DeadMG Mar 16 '17 at 15:10
21

The main problem is that it tightly couples the server to the client, who must know the HTML structure. It also makes the endpoints more difficult to re-use in new ways or new applications.

Returning plain data and letting the client render it decreases coupling and increases flexibility and testability- you can run unit tests on the client for mock data, and run unit tests on the server to test the desired data.

  • 11
    HTML can be made reasonably generic. You can return a bulleted list, for example, and stuff it into a div, where it will be subject to styling by the prevailing CSS. – Robert Harvey Mar 14 '17 at 16:16
  • 10
    That's not all that useful if I need to stuff it into a span this time. Or render it in another application which is not rendered in HTML. – DeadMG Mar 14 '17 at 19:50
  • 2
    I would rephrase as will always be composing HTML, and the form of that HTML must always be completely consistent across all usages, which is not a very helpful guarantee. For instance, in our app we have lists, but we actually didn't use ul and li but instead changed to making each one a div (don't remember why). Would have been tricky if the server returned a bunch of HTML with uls and lis in it. – DeadMG Mar 14 '17 at 21:25
  • 2
    Seems pointless to get the guarantee in the first place when you can just return the data and let the client render that as HTML as it sees fit (if it's even going to render it at all) – DeadMG Mar 14 '17 at 21:29
  • 1
    The only scenario I can see where returning HTML would be preferable is if the client doesn't have enough resources to do the rendering itself. – DeadMG Mar 14 '17 at 21:32
14

I think you have it a little backwards. You say:

we have no test whatsoever, so we have no particular reason to create this JSON endpoint

A reason to use a proper endpoint would be so that you could test it. I'd say that having no tests is a very good reason to start writing some. That is, if there is any logic that would be suitable to test.

200k lines of code is a lot to refactor and are probably hard to maintain. Breaking out some endpoints and testing them might be a good place to start.

Another reason might be to separate the server from the client. If, in the distant future, the application design or layout changes, it's easier to work with a proper data format than HTML output. In an ideal world, you would only have to change the client and not touch the server at all.

  • The point about layout changes sounds more like a need to separate templates from the underlying data, but there's no reason those templates have to be on the client. Indeed, there are plenty of reasons for them not to be, e.g. you can't decide to hide data from the client if your rendering is on the client. HTML partials can be re-skinned just fine if they're output by the same templating system as full HTML pages. – IMSoP Mar 15 '17 at 10:09
6

There are 3 ways (at least?) to build up a web page:

  • Generate the entire page server side.
  • Return a bare bones page from the server plus code (JavaScript), and have the page fetch its data and render into HTML client side.
  • Return a partial page plus code, and have the code fetch pre-rendered blocks of HTML that it can drop into the page.

The first one is fine. The second one is also fine. It's the last one that's the problem.

The reason is simple: you have now divided the construction of the HTML page into completely disconnected parts. The problem is one of maintenance. Now you have two separate entities in charge of managing the details of the UI. So you have to keep CSS and other similar details in sync between the two separate pieces. You changed the width of the side bar? Great. Now the HTML fragment that comes back causes horizontal scrolling because its assumptions about the side bar width no longer hold. You changed background color for that block? Great, now your HTML fragment's font color clashes because it assumed a different background color and someone forgot to test that endpoint.

The point is that you have now split up knowledge that should be centralized in a single place (namely the presentation logic), and this makes it more difficult to make sure all the pieces fit together correctly. By using a JSON API, you can instead keep all that logic in just the front end, or you can keep it all in your server side templates if you render your data into HTML to begin with. It's about keeping the presentation knowledge/logic in a single place, so it can be managed consistently and as part of a single process. HTML/CSS/JS is difficult enough to keep straight without breaking it up into a lot of tiny pieces.

JSON APIs also have the additional benefit of making the data available completely independently from the presentation logic. This allows multiple, different presenters, such as both a mobile app and a web page, to consume the same data. In particular, it enables consuming the data without a browser (such as mobile apps or nightly cron jobs); these consumers may not even be able to parse HTML. (This of course necessarily relies on having a situation where the data is the same between the different consumers, or one can use a subset of the other.) Whether you need this ability depends on your particular application's requirements, though, while managing your presentation logic is necessary regardless. I will say that if you implement it up front, though, you'll be better prepared for future growth.

  • 2
    I actually think avoiding duplicate display logic can be a good reason to render HTML page fragments: if you render part of the page on the server (e.g. the header and basic layout), then generate other parts based on JSON data on the client, you have two different sets of templates. Rendering partials on the server moves this logic back to your central presentation layer, which can use the same template to render an individual component as it would if it was assembling the whole page statically. – IMSoP Mar 15 '17 at 10:17
  • 1
    you're the only one who mentions mobile, I want to give you a thousand upvotes for that – Lovis Mar 15 '17 at 15:34
  • 1
    @IMSoP If you need a dynamic page, then you have to have front end logic. If you still render fragments server side, now you have to make sure the front end's assumptions match the pre-conditions of the server constructing the fragments. You can't break that dependency. Keeping that knowledge in sync is harder if it's split up into completely divided systems. If you just render on the front end, those assumptions are centralized. I think I'd also avoid mixing a dynamic front end with an initial state in a server side template; a "bootstrap" to start the front end is simpler. – jpmc26 Mar 16 '17 at 19:32
4

I would say there is nothing wrong with the server returning an HTML fragment and the UI assigning it to .innerHTML of some element. This is, in my opinion, the easiest way to develop asynchronous JavaScript code. The benefit is that as little as possible is done using JavaScript and as much as possible is done in a controlled back-end environment. Remember that JavaScript support in browsers varies but your back-end always has the same version of the back-end components, meaning that doing as much as possible in the back-end will mean as little as possible version incompatibilities.

Now, sometimes you want more than just an HTML fragment. For example, a status code and an HTML fragment. Then you can use a JSON object that has two members, statusCode and HTML of which the second can be assigned to .innerHTML of some element after checking the statusCode. So, using JSON and using innerHTML are by no means alternative exclusive approaches; they can be used together.

By using JSON you can even have multiple HTML fragments in the same response which get assigned to the .innerHTML of multiple elements.

In summary: do use .innerHTML. It makes your code compatible with as many browser versions as possible. If needing more, use JSON and .innerHTML together. Avoid XML.

4

There is nothing wrong in principle. The question is: What do you want to achieve?

JSON is perfect for transmitting data. If you send HTML instead and expect the client to extract the data from the HTML, that's rubbish.

On the other hand, if you want to transmit HMTL that is going to be rendered as HTML, then send it as HTML - instead of packing the HTML into a string, turning the string into JSON, transmitting JSON, decoding it on the other side, getting a string, and extracting the HTML from the string.

And just yesterday I ran into code that put two items into an array, turned the array into JSON, put the JSON into a string, put the string inside an array, turned the whole array into JSON, sent it to the client, which decoded the JSON, got an array containing a string, took the string, extracted the JSON from the string, decoded the JSON, and got an array with two items. Don't do that.

  • +1 Exactly. The first question is, what do you need to receive? There would be little wrong with an endpoint returning sidebar ads as a bit of HTML, or perhaps a footer, or similar elements. – SQB Mar 18 '17 at 10:36
3

This all depends on the purpose of the API, but generally what you describe is a pretty strong violation of separation of concerns:

In a modern application, API code should be responsible for data, and client code should be responsible for presentation.

When your API returns HTML, you are tightly coupling your data and presentation. When the API returns HTML, the only thing you can do (easily) with that HTML is display it as some part of a larger page. From a different angle, the only thing the API is good for is supplying your page with HTML. Plus, you've spread your HTML across both the client and server code. This can make maintenance a headache.

If your API returns JSON, or some other form of pure data, it becomes much more useful. The existing app can still consume that data, and present it appropriately. Now, though, other things can use the API to access the same data. Again, too, maintenance is easier - all the HTML resides in one place, so if you want to re-style the whole site, you have no need to change your API.

  • 5
    "In a modern application, API code should be responsible for data, and client code should be responsible for presentation." Why should this always be the case? I agree that this is a common pattern and that it makes certain things easier, but I see no reason to elevate it to the level of "should" ... It is a decision that needs to be taken on a case by case basis, and there are certainly reasons why in some situations you may want to make a different decision. – Jules Mar 14 '17 at 23:37
  • @Jules because if you have an API and a client, having both in charge of rendering is a violation of separation of concerns. (Now, you don't necessarily have an API and a client. You can have only one component, and have it do the whole presentation. But then you don't have an API) – njzk2 Mar 15 '17 at 2:20
  • @njzk2 Just because an API delivers HTML data does not mean it has rendered it. It may be treating the HTML as a blob and storing it in a database, for example. Also, some rendering may be required on the server rather than the client (eg providing static pages for search engines) so reusing that capability could be seen as elimination of duplication. – Jules Mar 15 '17 at 12:13
  • 1
    Also, it is entirely possible to produce a client and api pair where all rendering occurs at the server and the client merely plugs delivered HTML into predefined slots in it's DOM. Jquery has an entire module dedicated to this kind of client, which suggests to me they must be reasonably common. – Jules Mar 15 '17 at 12:20
  • 1
    @Jules lots of things are reasonably common, that's not a reason why they are reasonable. – njzk2 Mar 15 '17 at 16:12
2

HTML is tied to a specific design and use.

With HTML, if you want to change the page layout you need to change how the html is generated by the server call. Usually, that requires a back-end programmer. Now you have back-end programmers, who by definition aren't your best html writers, handling these updates.

With JSON, if the page layout changes, the existing JSON server call doesn't necessarily have to change at all. Instead, your front-end developer, or even the designer, updates the template to produce the different HTML you want from the same basic data.

Additionally, the JSON can become the basis for other services. You might have different roles that need to see the same basic data in different ways. For example, you may have a customer web site that shows data about a product in an order page, and an inside sales page for reps that shows the same data in a very different layout, perhaps alongside some other information not available to general customers. With JSON, the same server call can be used in both views.

Finally, JSON can scale better. In recent years we've kind of gone overboard with client-side javascript frameworks. I think it's time to actually take a step back and start thinking about what javascript we're using, and how it affects browser performance... especially on mobile devices. That said, if you're running a site large enough to require a server farm or cluster, instead of a single server, JSON can scale better. Users will give you processing time in their browsers for free, and if you take advantage of that you can reduce the server load in a large deployment. JSON also uses less bandwidth, so again, if you're big enough and use it appropriately, JSON is measurably cheaper. Of course, it can also scale worse, if you end up feeding 40KB libraries to parse 2KB of data into 7KB of html, so again: it pays to be aware of what you're doing. But the potential is there for JSON to improve performance and costs.

1

There is nothing wrong with such an endpoint if it fulfills its requirements. If it is required to spit out html that a known consumer can parse effectively, sure, why not?

The problem is that, for the general case, you want your endpoints to spit out a payload that is well-formed and effectively parse-able by a standard parser. And by effectively parse-able, I mean, parse-able declaratively.

If your client is forced to read the payload and pry open information bits from it with loops and if-statements, then it is not effectively parse-able. And HTML, being the way its, it is very forgiven in not requiring itself to be well-formed.

Now, if you make sure your html is xml-compliant, then you are gold.

With that said, I have a significant problem with this:

I'm gonna tell you this: an endpoint that returns HTML reduces your dependency on JavaScript libraries and reduces the load on the user browser since it won't need to interpret/execute JS code to create DOM objects - the HTML is already there, it's just a matter of parsing the elements and rendering them. Of course, this means we're talking about a reasonable amount of data. 10 megabytes of HTML data isn't reasonable.

That's a bad idea no matter how you cut it. Decades of collective industrial experience has shown us that, in general, it is a good idea to separate data (or model) from its display (or view.)

Here you are conflating the two for the purpose of speedy JS code execution. And that's a micro optimization.

I've never seen this as a good idea except in very trivial systems.

My advise? Don't do it. HC SVNT DRACONES, YMMV, etc.

0

JSON is just a textual presentation of structured data. A client naturally needs to have a parser to process data but virtually all languages have JSON parser functions. It is much more efficient to use JSON parser than to use HTML parser. It takes little footprint. Not so with an HTML parser.

In PHP, you just use json_encode($data) and it's up to the client on the other side to parse it. And when you fetch JSON data from a web service, you just use $data=json_decode($response) and you can decide how to use data like you would do with variables.

Suppose you develop an app for a mobile device, why do you need the HTML format when mobile apps rarely use the web browser to parse data? Many mobile apps use JSON (most common format) to exchange data.

Considering that mobiles are often on metered plans, why do you want to use HTML that takes much more bandwidth than JSON?

Why use HMTL when HTML is limited in its vocabulary and JSON can define data? {"person_name":"Jeff Doe"} is more informative than HTML can provide about its data since it only defines structure for HTML parsers, not define data.

JSON has nothing to do with HTTP. You can put JSON in a file. You can use it for configurations. Composer uses JSON. You can use it to save simple variables to files as well.

0

It's difficult to categorize a right or a wrong. IMO, the questions I'll ask are: "should it", or "can it do with less?".

Every endpoint should strive for communicating with as little content as possible. The signal-to-noise ratio is typically HTTP codes < JSON < XHTML. In most of the situations, it's good to chose the least noisy protocol.

I differ on the point about client browser load by @machado, as with modern browsers this is a non-issue. Most of them are equipped in dealing with HTTP codes and JSON responses pretty well. And although you don't have tests at the moment, long term maintenance of a less noisy protocol would be cheaper than the one above it.

protected by gnat Mar 15 '17 at 13:49

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.