How do searches fit into a RESTful interface?

Question

When designing a RESTful interface, the semantics of the request types are deemed vital to the design.

• GET - List collection or retrieve element
• PUT - Replace collection or element
• POST - Create collection or element
• DELETE - Well, erm, delete collection or element

However, this doesn't seem to cover the concept of "search".

E.g. in designing a suite of web services that support a Job Search site you might have the following requirements:

• Get individual Job Advert
  • GET to domain/Job/{id}/
• Create Job Advert
  • POST to domain/Job/
• Update Job Advert
  • PUT to domain/Job/
• Delete Job Advert
  • DELETE to domain/Job/

"Get All Jobs" is also simple:

• GET to domain/Jobs/

However, how does the job "search" fall into this structure?

You could claim it's a variation of "list collection" and implement as:

• GET to domain/Jobs/

However, searches can be complex and it's entirely possible to produce a search that generates a long GET string. That is, referencing a SO question here, there are issues using GET strings longer than about 2000 characters.

An example might be in a faceted search - continuing the "job" example.

I may allow for searching on facets - "Technology", "Job Title", "Discipline" as well as free-text keywords, age of job, location and salary.

With a fluid user interface and a large number of technologies and job titles, it is feasible that a search could encompass a large number of facet choices.

Tweak this example to CVs, rather than jobs, bring in even more facets, and you can very easily imagine a search with a hundred facets selected, or even just 40 facets each of which are 50 characters long (e.g. Job Titles, University Names, Employer Names).

In that situation it might be desirable to move a PUT or POST in order to ensure that the search data will get correctly sent. E.g.:

• POST to domain/Jobs/

But semantically that's an instruction to create a collection.

You could also say you'll express this as the creation of a search:

• POST to domain/Jobs/Search/

or (as suggested by burninggramma below)

• POST to domain/JobSearch/

Semantically it may seem to make sense, but you're not actually creating anything, you're making a request for data.

So, semantically it's a GET, but GET isn't guaranteed to support what you need.

So, the question is - Trying to keep as true to RESTful design as possible, whilst ensuring that I'm keeping within the limitations of HTTP, what is the most appropriate design for a search?

Answer 1

You should not forget that GET requests have some superior advantages over other solutions:

1) GET requests can be copied from the URL bar, they are digested by search engines, they are "friendly". Where "friendly" means that normally a GET request should not modify anything inside your application (idempotent). This is the standard case for a search.

2) All of these concepts are very important not just from user and search engine, but from an architectural, API design standpoint.

3) If you create a workaround with POST/PUT you will have problems which you are not thinking of right now. For example in case of a browser the navigate back button / refresh page / history. These can be solved of course, but that's going to be another workaround, then another and another ...

Considering all this my advice would be:

a) You should be able to fit inside your GET with using clever parameter structure. In extreme case you can even go for tactics like this google search where I set a lot of parameters still its a super short url.

b) Create another entity in your application like JobSearch. Assuming you got so much options, its probable that you will need to store these searches as well and manage them, so its just clearing up your application. You can work with the JobSearch objects as a whole entity, meaning you can test it / use it easier.

---

Personally I would try to fight with all my claws to get it done with a) and when all hope is lost, I would crawl back with tears in my eyes to option b).

Answer 2

TL;DR: GET for filtering, POST for searching

I make a distinction between filtering the results from listing a collection vs a complex search. The litmus test I use is basically if I need more than filtering (positive, negative, or range) I consider it a more complex search requiring POST.

This tends to be reinforced when thinking about what will be returned. I usually only use GET if a resource has a mostly complete life-cycle (PUT, DELETE, GET, collection GET). Typically in a collection GET I'm returning a list of URIs which are the REST resources that make up that collection. In a complex query I may be pulling from multiple resources in order to construct the response (think SQL join) so I won't be sending back URIs, but actual data. The problem is the data will not be represented in a resource, so I will always have to return data. This seems to me a clear cut case of requiring a POST.

--

It's been awhile and my original post was a bit sloppy so I thought I would update.

GET is the intuitive choice for returning most kinds of data, collections of REST resources, structured data of a resource, even singular payloads (images, docs, etc).

POST is the catchall method for anything that doesn't seem to fit under GET, PUT, DELETE, etc.

At this point I think simple searches, filtering do intuitively make sense through GET. Complex searches is up to personal preference especially if you are throwing in aggregation functions, cross correlations (joins), re-formatters, etc. I would argue the GET params should not get too long, and it a largish query (however it's being structured) can often make more sense as a POST request body.

I also consider the experience aspect of API usage. I generally want to make most of the methods as easy to use and intuitive as possible. I'll push calls that are more flexible (and therefore more complex) into POSTs and on a different resource URI, especially if it's inconsistent with the behavior of other REST resources in the same API.

Either way, consistency is probably more important than whether you are doing search in GET or POST.

Hope this helps.

Answer 3

In REST, resource definition is very broad. It's really however you want to bundle some data.

• It's useful to think of a search resource as a collection resource. The query parameters, sometimes called the searchable portion of the URI, narrow the resource down to the items the client is interested in.

For example, the main Google URI points to a collection resource of "links to every site on the Internet". Query parameters narrow that to the sites you want to see.

(URI = universal resource identifier, of which URL = universal resource locator, where the familiar "http://" is the default format for a URI. So URL is a locator, but in REST its good to generalize that to a resource identifier. People use them interchangably, though.)

• Since the resource you're searching on in your example is the jobs collection, it makes sense to search with

GET site/jobs?type=blah&location=here&etc=etc

(return) {jobs: [{job: ...}]}

And then use POST, which is the append or process verb to add new items to that collection:

POST site/jobs

{job: ...}

• Note that it's the same structure for the job object in each case. A client can GET a collection of jobs, using query params to narrow the search, and then use the same format for one of the items to POST a new job. Or it can take one of those items and PUT to its URI to update that one.

• For really long or complicated query strings, convention makes it OK to send those as POST requests instead. Bundle the query paramters up as name/value pairs, or nested objects in a JSON or XML structure and send it in the body of the request. For example, if your query has nested data instead of a bunch of name/value pairs. The HTTP spec for POST describes it as the append or process verb. (If you want to sail a battleship through a loophole in REST, use POST.)

I'd use that as the fallback plan, though.

What you lose when you do that though is a) GET is nullipotent -- that is, it doesn't change anything -- POST is not. So if the call fails, middleware won't automatically retry or cache results, and 2) with the search parameters in the body, you can't cut and paste the URI anymore. That is, the URI isn't a specific identifier for the search you want.

To differntiate between "create" from "search". There are a couple of options that are consistent with REST practice:

• You could do it in the URI by adding something to the name of the collection, like job-search instead of jobs. That just means you're treating the search collection as a separate resource.

• Since the semantics of POST is both append OR process, you could identify search bodies with the payload. Like {job: ...} vs. {search: ...}. It's up to the POST logic to post or process it appropriately.

That's pretty much a design/implementation preference. I don't think there's a clear convention.

So, like you've already laid out, the idea is to define a collection resource for jobs

site/jobs

Search with GET + query params to narrow the search. Long or structured data queries go into the body of a POST (possibly to a separate search collection). Create with POST to append to the collection. And update with PUT to a specific URI.

(FWIW the style convention with URIs is to use all lowercase with words separated by hyphens. But that doesn't mean you have to do it that way.)

(Also, I should say that from your question, it's clear that you're a long way down this road. I spelled things out kind of explicitly just to line them up, but your question had already addressed most of the semantic issues in this answer. I was just lacing it with some convention and practice.)

Answer 4

I generally use OData queries, they operate as a GET call but allow you to restrict the properties which are returned and filter them.

You use tokens such as $select= and $filter= so you will end up with a URI which looks something like this:

/users?$select=Id,Name$filter=endswith(Name, 'Smith')

You can also do paging using $skip and $top and ordering.

For more information, check out OData.org. You haven't specified which language you are using, but if it's ASP.NET, the WebApi platform supports OData queries - for others (PHP etc) there are probably libraries you can use to translate them into database queries.

Answer 5

This is an old answer but I still can contribute a bit to the discussion. I've observed very often a misunderstanding of REST, RESTful and Architecture. RESTful doesn't ever mention anything about NOT building search, there's nothing in the RESTful about architecture, it's a set of design principles or criteria.

To describe a Search in a better way we have to talk about an architecture in particular and the one that fits better is Resource Oriented Architecture (ROA).

In RESTful there are principles to design, idempotent doesn't mean the result cannot change as I read in some answers, it means the result of an independent request doesn't depend on how many times is executed. It can change, let's imagine I'm continuously updating a database feeding it with some data that is served by a RESTful Api, executing the same GET might change the result but it doesn't depend on how many times it's been executed. If I'm able to freeze the world it means there's no state, transformation, anything inside of the service when I request the resource that leads to a different result.

By definition, a resource is anything that's important to be referenced as a thing by itself.

In a resource oriented architecture (let's call it ROA from now on for brevity) we focus on the resource that could be a lot of things:

• A version of a document
• The last updated version of the document
• A result coming from a search
• A list of objects
• The first article I bought from an e-commerce

What it makes it unique in terms of resource is the addresability which means it has only one URI

In that way the search fits perfectly in RESTful considering ROA. We have to use GET because I'm assuming your search is a normal search and it doesn't change anything, so it's idempotent (even if it returns different things depending on new elements added). There's a confusion here in that way because I could stick to RESTful and not to ROA, it means I could follow a pattern that creates a search and return different things with the same parameters because I'm not using the addressability principle of ROA. How is that? Well, if you send the search filters in the body or header the resource is not ADDRESSABLE.

You can find the principles of what's exactly and URI in the W3 original document:

https://www.w3.org/DesignIssues/Axioms

Any URL in this architecture has to be self-descriptive. It's necessary if you follow the principles to address everything in the URI, it means you can use / (slash) to separate whatever you need or query parameters. We know there are limitations on that but this is the architecture pattern.

Following the ROA pattern in RESTful a search is not more than any other resource, the only difference is the resources come from a computation instead of a direct relation to the object itself. Based on the principle I could address and obtain a simple arithmetic calculation service based on the following pattern:

http://myapi.com/sum/1/2

Where sum, 1 and 2 can be modified but the result of the computation is unique and it's adressable, every time I call with the same parameters I obtain the same and nothing changes in the service. The resouce /sum/1/2 and /substract/5/4 stick to the principles perfectly.

Answer 6

One approach to consider is treating the set of possible queries as a collection resource, e.g. /jobs/filters.

POST requests to this resource, with the query parameters in the body, will either create a new resource or identify an existing equivalent filter and return a URL containing its ID: /jobs/filters/12345.

The id can then be used in a GET request for jobs: /jobs?filter=12345. Subsequent GET requests on the filter resource will return the definition of the filter.

This approach has the advantage that it frees you from the query parameter format for filter definition, potentially providing you with more power to define complex filters. OR conditions are one example that I can think of that are difficult to accomplish with query strings.

A drawback to this approach is that you lose readability of the URL (although this can be mitigating by retrieving the definition though a GET request for the filter resource). For this reason you may also want to support the same or a subset of the query parameters on the /jobs resource as you would support for a filter resource. This could be used for shorter queries. If this feature is provided, in order to maintain cacheability between the two types of filtering, when using query parameters on the /jobs resource, the implementation should internally create/reuse a filter resource and return a 302 or 303 status indicating the URL in the form of /jobs?filter=12345.

Answer 7

GET is ok, if you have a static collection that always returns the same results (representation) for one URI. That also implies that the data generating these representations is never altered. The source is a read-only database.

Having GET return different results for one and the same URI violates idempotency/safety and the CoolURI principle and is consequently not RESTful. It is possible to have idempotent verbs write to a database, but they must never affect the representation.

A common search starts with a POST request that returns a reference to the result. It generates the result (it is new and can be fetched with a subsequent GET). This result can be hierarchical (further references with URIs that you can GET), of course, and might reuse elements of earlier searches, if it makes sense for the application.

By the way, I know people do it differently. You don't need to explain to me how convenient it can be to violate REST.