1

I struggle to understand Roy Fielding’s REST alternative to HTTP cookies, § 6.3.4.2 ‘Cookies’ and § 6.5.4.1 ‘Application State in a Network-Based System’ of his doctoral dissertation Architectural Styles and the Design of Network-Based Software Architectures. How do you interpret it?

6.3.4.2 Cookies

An example of where an inappropriate extension has been made to the protocol to support features that contradict the desired properties of the generic interface is the introduction of site-wide state information in the form of HTTP cookies [73]. Cookie interaction fails to match REST's model of application state, often resulting in confusion for the typical browser application.

An HTTP cookie is opaque data that can be assigned by the origin server to a user agent by including it within a Set-Cookie response header field, with the intention being that the user agent should include the same cookie on all future requests to that server until it is replaced or expires. Such cookies typically contain an array of user-specific configuration choices, or a token to be matched against the server's database on future requests. The problem is that a cookie is defined as being attached to any future requests for a given set of resource identifiers, usually encompassing an entire site, rather than being associated with the particular application state (the set of currently rendered representations) on the browser. When the browser's history functionality (the "Back" button) is subsequently used to back-up to a view prior to that reflected by the cookie, the browser's application state no longer matches the stored state represented within the cookie. Therefore, the next request sent to the same server will contain a cookie that misrepresents the current application context, leading to confusion on both sides.

Cookies also violate REST because they allow data to be passed without sufficiently identifying its semantics, thus becoming a concern for both security and privacy. The combination of cookies with the Referer [sic] header field makes it possible to track a user as they browse between sites.

As a result, cookie-based applications on the Web will never be reliable. The same functionality should have been accomplished via anonymous authentication and true client-side state. A state mechanism that involves preferences can be more efficiently implemented using judicious use of context-setting URI rather than cookies, where judicious means one URI per state rather than an unbounded number of URI due to the embedding of a user-id. Likewise, the use of cookies to identify a user-specific "shopping basket" within a server-side database could be more efficiently implemented by defining the semantics of shopping items within the hypermedia data formats, allowing the user agent to select and store those items within their own client-side shopping basket, complete with a URI to be used for check-out when the client is ready to purchase.

6.5.4.1 Application State in a Network-based System

REST defines a model of expected application behavior which supports simple and robust applications that are largely immune from the partial failure conditions that beset most network-based applications. However, that doesn't stop application developers from introducing features which violate the model. The most frequent violations are in regard to the constraints on application state and stateless interaction.

Architectural mismatches due to misplaced application state are not limited to HTTP cookies. The introduction of "frames" to the Hypertext Markup Language (HTML) caused similar confusion. Frames allow a browser window to be partitioned into subwindows, each with its own navigational state. Link selections within a subwindow are indistinguishable from normal transitions, but the resulting response representation is rendered within the subwindow instead of the full browser application workspace. This is fine provided that no link exits the realm of information that is intended for subwindow treatment, but when it does occur the user finds themself viewing one application wedged within the subcontext of another application.

For both frames and cookies, the failure was in providing indirect application state that could not be managed or interpreted by the user agent. A design that placed this information within a primary representation, thereby informing the user agent on how to manage the hypermedia workspace for a specified realm of resources, could have accomplished the same tasks without violating the REST constraints, while leading to a better user interface and less interference with caching.

  • How do we carry user preferences stored in a URI across multiple actions?
    Example. — I assume that Roy Fielding meant that the server should propagate user preferences in the hyperlinks of response representations: from the HTML representation in light mode of the URI /preferences of a website, a user selects a dark theme button which is an hyperlink to the URI /preferences?theme=dark. The server responds with an HTML representation in dark mode with hyperlinks to URIs updated with an additional query component theme=dark. So if the user then selects the website logo which is an hyperlink to the URI /home?theme=dark (not /home), the server responds again with an HTML representation in dark mode with hyperlinks to URIs updated with an additional query component theme=dark.
  • How do we embed shopping items in the HTML hypermedia format?
    Example. — I assume that Roy Fielding meant that HTML should define a new <shopping> element for representing shopping items.
    <!DOCTYPE html>
    <html lang="en">
    <title>Shop</title>
    <h1>Our products</h1>
    <shopping name="beer" stock="4" price="5.99" currency="USD">
      <quantity>1<quantity/> Beer (USD 5.99)
    </shopping>
    <shopping name="cider" stock="1" price="3.99" currency="USD">
      <quantity>0<quantity/> Cider (USD 3.99)
    </shopping>
    <shopping name="wine" stock="3" price="9.99" currency="USD">
      <quantity>0<quantity/> Wine (USD 9.99)
    </shopping>
    </html>
    
  • How do we store user shopping cart in the client without Javascript?
    Example. — I assume that Roy Fielding meant that if the HTML hypermedia format supported a new <shopping> element for representing shopping items, browsers would understand them natively and therefore would be able to update their application states according to user interactions with the <shopping> elements, instead of relying on client-side scripting.
17
  • 1
    Just so that we're on the same page, be aware that Roy Fielding’s REST has almost nothing to do with what the software industry calls REST; he's not talking about a better way to make web services, he's describing the architecture of the Internet, in a generalized way. His viewpoint is that of a protocol developer. When he says 'application', he means one that operates within the rules and constraints of the architecture and the application-level protocols of the Web. Web developers don't necessarily want to be too constrained by that, opting to treat the Web more as a delivery mechanism. 1/3 – Filip Milovanović May 25 at 6:31
  • 1
    So, you see, there are somewhat conflicting interests, which, I'm guessing, lead to protocols being augmented with features (like cookies and frames) that were not quite in line with the original vision/architecture. I'm not 100% certain I'm correct, but I don't think he's necessarily proposing a better alternative (i.e., it's not meant to be practical now), I think he's describing what would have been more historically desirable, from the perspective of the protocol community. 2/3 – Filip Milovanović May 25 at 6:31
  • 2
    He seems to be saying that these features should have been somehow made part of the protocols in a way that lets the browser / user agent understand what's going on (e.g. like it understands the semantics of HTML). Notice, for example, that one of his complaints is about cookies being opaque to the browser (because it can't understand proprietary, application-specific formats). This would require people to agree on a set of standards for these things, though. Besides, application vendors would still want the ability to operate outside of these constraints, to offer flashy features 3/3 – Filip Milovanović May 25 at 6:32
  • 5
    HTTP was initially designed as a stateless protocol where any application state is represented in the URL, and the client change state by following URLs. Cookies muddied this by introducing a separate session state. For example pressing the back button should always bring you back to the previous state, but this is not necessarily the case with cookies. If I send a link to you, you might not actually see the same page or resource since you don't have the same cookie. – JacquesB May 25 at 8:35
  • 1
    Perhaps you should indicate what parts you struggle to understand, since I just end up repeating what the quote says. – JacquesB May 25 at 8:42
1

Just to disambiguate things a bit. "Application State" in REST is used to describe the state of the conversation taking place. So how far are you in your order process, on which page are you in the "wizard" or things like that.

What constitutes "Application State" and just plain "Server State" is sort-of fluid. Your example about "theme" is one of these things. You can design that as "Application State" (to be transient for the server), but it can be easily defined as "Server State", that is, stored an represented on the server (you know, stored on your profile, or something like that).

So there is nuance to this. With that in mind, I'll try to answer some of your questions:

How do we carry user preferences stored in a URI across multiple actions?

You can use both the URI and the content for that. One pretty obvious way is using "hidden" input fields for example, if you're dealing with forms. This was (is?) a pretty common way to deal with "wizard" style multi-page processes.

How do we embed shopping items in the HTML hypermedia format?

Again, you can use URIs and hidden input fields, but you more than likely want some more persistent solution, since you don't want items to disappear when the user hits the 'back' button.

Local storage is also one option. Making it essentially "Client State", which is a third thing separate from "Application State" and "Server State".

The client is also free to have its own state. Like when I use Amazon and I want to buy 2 things, I'll keep the progress in my head and order when I found both items. That is client state.

Again, the thing that is not cool, is to keep "Application State" on the server (or anywhere else other than the representations the client currently sees), since our goal is that the server can process each request completely on its own, independently of any previous messages, and that the client can control the "Application State" exclusively by choosing server provided options in the current representations it sees.

How do we store user shopping cart in the client without Javascript?

You can use URI rewriting, and/or hidden inputs. Or you can make it an "official" server state, and have proper resources for it. Or you can use Javascript and local storage, that is also not against Fielding's REST. It is in fact explicitly mentioned that the server can push code onto the client to extend its capabilities.

What you can't (shouldn't) do according to REST is have a Cookie which is essentially a pointer to a server-side application state, and/or is an opaque application state the client doesn't know about, therefore can't/doesn't control.

I hope I managed to clear up some points.

11
  • Thanks a lot Robert, this is both very thorough and very clear, I am really impressed by the quality of your answer. I especially appreciate the client state/application state distinction as it anticipates the question I was about to ask you. The Web storage API is client state and HTTP Cookies are either references to server-side application state or opaque client state—you made a typo?—(since they are independent of the browser’s history and this is why Fielding does not like them). – Maggyero May 28 at 1:00
  • None are client-side application state like Fielding requires, so the only Javascript solution seems to be the History API. I actually opened a question on this matter and nobody gave a satisfying answer so far (if you are interested). – Maggyero May 28 at 1:00
  • @Maggyero That's not a typo, cookies are not client state, because the client doesn't know what it is. It is opaque. Cookies are Application State (i.e. "Session" State). If the cookies wouldn't be submitted to the server, but used by the client to pick options the server provides, that would be client state, and would be completely fine. I'll take a look at your other question. – Robert Bräutigam May 28 at 7:19
  • 1
    Fielding talks about including userid in all URIs for the purpose of tracking the user on the server. It's the same thing that Java Servlets did with the "sessionId" parameter that was present when cookies were not available. They are tracking session state on the server, which is clearly not stateless. And they make shared caching impossible as a bonus. None of those is an issue for server-side carts. Those are valid resources, as such part of the "server state", with whatever URIs the server deems appropriate. – Robert Bräutigam May 31 at 8:05
  • 1
    I don't think I contradict that answer. It's not the userid itself that is the problem, but how you use it. If it is part of a proper server resource, it is not a problem. If it is used as a session state key, it is a problem. It's your shopping cart after all. Sending a link to someone else, caching it or bookmarking it will work as expected. Again, because it is a proper resource, i.e. "server state". It is not as clear cut as "no userid shall ever be part of any URIs". That is not what Fielding is saying in my interpretation. – Robert Bräutigam May 31 at 10:32
1

HTTP was initially designed as a stateless protocol, which means a server should not need to maintain any client-specific session state between requests. Instead all information necessary to process a given request should be included in the request itself. State between requests should only exist on the client side.

The cookies protocol was introduced to make it possible to track client-specific state on the server, for example to maintain a shopping basket or preferences between request. But Roy Fielding argues the design of the cookie protocol is not the best and breaks several of the core design principles behind the web.

So what does he propose instead? He describes two scenarios which are often implemented using cookies and describe how they could be implemented in a way which does not break the core principles.

The first is the use of cookies for user-specific preferences, e.g choice of language or (as in your example) theme. Fielding describes how this can be supported using just URLs.

A web site (or the web as a whole) could be considered a state machine where each state is identified by an URL. The client navigate this state graph by following links. You can always go back to a previous state by using the back button. We usually think of this as navigating pages, but any state change (e.g. selecting a language or theme) could be implemented as following links. Your example with dark theme describes this pretty well.

The second example is a shopping cart. Fielding recognizes that encoding a shopping cart state in URLs is not ideal. He doesn't say why, but I imaging because it is rater counter intuitive that going to a product page, add a product, and then press the back button will remove the product again rather then return to the previous page. Bookmarking a page would also bookmark the exact state of the shopping basket at that point, which would be rather weird.

So there is a need for maintaining state separately from the state encoded in in the URL. But he argues that such a state should be purely client side. The server would not track the current state of the shopping basket per user. Only when the user is ready to order would the whole cart state be passed to the server in a POST request.

But he does not describe in detail how such a technique could be implemented, since his intention in the paper is just to analyze the current state of affairs, not to propose any new protocols. He is not even talking about HTML specifically.

Today a client-side shopping-basket could be implemented using a SPA or using LocalStorage. But neither was realistically available 20 years ago when Fielding wrote the paper. So he is not proposing people stop using cookies, he is just describing some issues with the design. It was still (at least at that point in time) the best solution which was actually available.

How do we store user shopping cart in the client without Javascript?

You can't. There is currently no mechanism which allows you to save state (apart from navigation state) on the client without the use of scripting.

It is possible that Fielding had a hypothetical feature in mind which did not rely on scripting. This would instead require some built-in support in the browser which does not currently exist. But this is pure speculation since he does not describe any concrete solution but only the constraints which such a solution should fulfill.

But Fielding is is not averse to scripting on the client side and does not see JavaScript as in conflict with REST.

3
  • Thanks Jacques for clarifying Fielding’s writings Jacques. Robert suggested that an alternative is to store the shopping cart in the server state (a.k.a. resource state): ‘Or you can make it an "official" server state, and have proper resources for it.’ But now comes the question of identification: should the user identifier be stored in the URI or in the Authorization HTTP request header field? – Maggyero May 28 at 16:01
  • @Maggyero I think you should ask this as a separate question since it seem only tangential to the Fielding quote, and more people will see the question then. – JacquesB May 28 at 16:24
  • The provided link is actually my question. – Maggyero May 28 at 16:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.