4

I am a sole developer and I recently wrote a new web application in the form of a API with Swagger and front-end. This was customers could use the API on their own, usually forscripting.

It is nearing completion, and my boss who is not a developer brought up that the API looks too difficult for the consumers, and he feels like there will be some push-back. For example, when you hit /api/customers you may get back related data in the form of Ids: { name: "Bob", FavoriteProducts: [ 1, 2, 5] }.

This works great for the front-end. There are many pages that load the related data on demand, sometimes we show a customer summary for example and don't use the FavoriteProducts, sometimes we show a huge table of customers so all that related data would take forever to load.

He says, the customers will not want to have to make separate calls for the related data, and he says on an insert it would be easier if they could provide product names and have them mapped to their Ids and persisted magically on the back end: { name: "Bob", FavoriteProducts: [ "Plastic Spiders", "Candy", "Pumpkins"] }. This sounds innocent enough but this is a gross simplifications of a lot more data and a lot more endpoints, lookup tables, code tables, etc.

We haven't had our first consumer yet, so we don't know if this will actually be a problem, but he said from his experience scripting, he wants to make as few API calls as possible.

I don't fully agree, I do think we should strive to make our customers lives easier, especially since it is a competitive world out there. But I also worked hard to make a RESTful API with good performance and well documented via Swagger which is consumable by Swagger client generators (like NSwag) who can read the docs and spit out a client in almost any language. I think there's a line to balance when it comes to adding a bunch of extra back-end code and technical debt because someone does not want to make 2 API calls.

I am guessing GraphQL could have made this easier, but I think this boss would have said it would be too hard for the scripting customers to learn this new language, and he wouldn't want to spend the time for me to learn it because he vetoed Angular for this project for that exact reason. Plus I didn't think this would be an issue. We re-wrote a large legacy app that stored all of its data in a single table to a RESTful API with proper lookup tables and data validation. So I assumed it would complicated consumption a bit and i figured that would be fine.

  • 1
    I don't quite understand - why would your API even matter to your users if you have a front end? – Aleksander Oct 24 '18 at 15:13
  • It's important to know your audience. In other words, are developers going to be using your API or are just typical users? I think an argument could be said for making things as straightforward as possible in either case, and ids aren't straightforward. This isn't really an issue if you have a frontend though. If you can't come up with a clean and persuasive argument to use ids, then my guess is you shouldn't try to persuade your boss either. – Neil Oct 24 '18 at 15:18
  • 1
    @Aleksander we have a customer or two who needs to script access to the customer data for integration/alerting purposes. There is a million reasons why someone would need API access to data. – Victorio Berra Oct 24 '18 at 15:23
  • @Neil Thanks. For now, its 90% normal usage through the front-end, and 10% scripted usage or automation. For example, people might make a script to create a new customer with pre-loaded favorites for some reason. This is definitely about knowing the customers. And being fast to market, being flexible and being able to make changes quickly, and keeping the tech debt low. – Victorio Berra Oct 24 '18 at 15:26
  • 2
    @VictorioBerra I wrote a web application which, by request of my boss, allowed regular expressions to be used as a more sophisticated means of searching. Most users were confused and didn't understand how to use it, and the ones who did know were already experienced with regular expressions. In other words, if I had to guess, developers are ultimately going to be your primary "user". I would still make the interface as readable as possible. Have you done benchmarks to demonstrate that ids are significantly faster? I don't think there is any doubt that it is faster, but is it important? – Neil Oct 24 '18 at 15:30
9

If you've ever used RESTful APIs, you've come across at least one where you have to make multiple API calls to get all the data you want. It just kinda comes with the territory. The benefits of a granular API are that you can get exactly what you need and nothing you don't. The cost is multiple calls, which can get bad if you need to build a deep object graph.

The flip side is to RETURN EVERYTHING. Favorite products, colors, upcoming sales, shipping details to Madagascar, etc. It's plenty friendly to read, but can be a pain to sift through when you only want the customer name and address. It also can have performance problems, especially with returning so much data.

So what do you do?

My first recommendation would be to stick with a pretty simple API like you have. Most developers won't be caught off guard by this. On top of that, if you don't have any customers to tell you what they want, how can you know what they want? You can guess, but as anyone who's developed software for a while can tell you, guessing what a customer wants only works out some of the time. Most of the time you get it wrong (or at least, not entirely right).

If your boss is insistent, you could provide a kind of compromise by allowing API users to request what data they get back. Say you have the following endpoint and response:

/customers/1 -> { name: "bob", favoriteProducts: [ 1, 3 ] }

Now say you add a parameter to say you want the full product information back:

/customers/1?favoriteProducts=full -> { name: "bob", favoriteProducts: [ { id: 1, name: "plastic spiders", ... }, { id: 3, name: "pumpkins", ... }] }

You can add as many configuration options as make sense. Just make sure to fully document them so there aren't any surprises.

  • Two examples of this pattern: Jenkins lets you specify the depth of the object graph (?depth=2) and Jira indicates which properties can be expanded (?expand=widgets). – Dan Wilson Oct 24 '18 at 17:14
  • It's an interesting idea to have a data agnostic depth like that. – Victorio Berra Oct 24 '18 at 17:39
  • 1
    With not knowing what customers want, even when you have customers, they'd still probably have their own wishes, so I'm not sure if it'd help. The boss (Product Owner) however, is in the position of arbritating all the different requirements and prioritise them. I'd just becareful of insisting the easy route because, as a developer, I know which one is easier to write. At the same time, I know which one is easier to consume. Also, adding parameter is not the best way and will clutter the API – imel96 Oct 25 '18 at 0:27
3

You are not tailoring your API to customers. Your boss wants you to tailor the API to his wild guess what customers might want.

As a customer of APIs (software developer, so I’m your customer, not the people your boss wants to sell to): The worst ever is an API that tries to be clever. Don’t write an API that tries to be clever. Unique IDs are an excellent way to access an API.

If customers are told by their developers that they don’t like the API then you can implement things - after your boss makes sure the customer pays. Remember the iron rule of a successful business: The customer is always right - if the customer pays.

1

I don't think that there is much to gain by opposing the boss here. He is the one who does the business decisions and whose job it is to anticipate the needs of his customers.

If following the patterns your boss suggests would make your standard front-end harder to create, maintain and/or hurt its performance, then you might want to consider to create two versions of every API endpoint. One "machine-friendly" version tailored for the requirements of the frontend and one "user-friendly" version tailored for the needs of scripters at the customer.

Creating this separate API in addition to your current one would likely not be any more work for now because you don't need to change anything on the already working client-side. But keep in mind that it will increase maintenance and documentation overhead in the long term. You might be able to reduce that overhead by sharing as much code as possible between them. You could implement one endpoint by calling the other.

  • 1
    I don't think "create two versions of every API endpoint" meshes well with OP's "I am a sole developer" – Dan Wilson Oct 24 '18 at 17:16
  • 1
    There are clear gains in having a single API, and clear gains in having it based on ids. OP's role as a sole developer is clearly not only implementation, but the design as well, and he is paid for his knowlegde in this area. Blindly following boss' requests, without a pushback, would be at least borderline unprofessional, even if eventually the boss has a point. – Frax Oct 24 '18 at 21:19
1

For example, when you hit /api/customers you may get back related data in the form of Ids: { name: "Bob", FavoriteProducts: [ 1, 2, 5] }.

This works great for the front-end. There are many pages that load the related data on demand, sometimes we show a customer summary for example and don't use the FavoriteProducts, sometimes we show a huge table of customers so all that related data would take forever to load.

Your boss has a use case here and your concern about performance is also valid. I don't think they are conflicting and addressing them both will make a better API.

For the API, I have to say that if I do a GET /api/customers/ I prefer to see a response like { name: "Bob", FavoriteProducts: [ "Plastic Spiders", "Candy", "Pumpkins"] }

On the performance side, I know you did some benchmark from comments, I think you need to profile your differently. The reason for that is the performance issues are there to be fixed, they shouldn't shape the form of the API (within reasons, of course).

One way to see this is to put sample responses in files and have a server serve these static files so you can see the difference between the two API style. I.e. :

GET /api/customers   <-- returns fav product IDs
GET /api/favorite_products/1
GET /api/favorite_products/2
GET /api/favorite_products/5

vs.

GET /api/customers <-- this one returns related fav products

I didn't try that myself but pretty sure the second one will be faster because of less round trips. This is what most customers would want. Even if I don't need the fav products, doesn't bother me, it's still only one call.

Of course, when implemented with all the extra look ups there will be performance hit and have to be addressed. It is not impossible to remove most of the extra load, e.g. by storing data differently, using in-memory key-value store, caching etc.

0

Option C. Best of both worlds.

To help think about this problem I'd like to express this idea in terms of entity relationships, which you can think of as database tables if it is easier.

The model

Based on this example...

{ name: "Bob", FavoriteProducts: [ 1, 2, 5] }.

...I am inferring the following entities:

  1. Customer, with a primary key of CustomerID
  2. Product, with a primary key of ProductID
  3. CustomerProduct, with a compound key of CustomerID + ProductID, which when taken separately also act as foreign keys to their respective tables.

The way you want to do it

You want two API calls.

  1. One call returns a single row from Customer and several rows from CustomerProduct
  2. One call returns a single row from Product

The way your boss wants to do it

  1. One call that returns data from Customer, CustomerProduct, and Product.

Option C. My recommendation

This is just like yours-- two calls-- but dividing the entities differently.

  1. One call to get a single Customer
  2. One call to get that customer's entire CustomerProduct list and any associated records from Product

This is a much more natural way to divide the work and much easier for a client to use-- they either want the products or not, and the IDs by themselves are useless.

This will also reduce the number of calls your web server will need to handle, and will reduce the load on your database, since the database will be able to use a single index seek followed by a single index scan to get the entire list. If the client asked for each product one by one, this would be impossible.

Finally, this design will get rid of some nasty concurrency/consistency bugs that could occur. For example, what if, in the middle of retrieving the list of a customer's products, something in the list changed, or one of the products was discontinued? While rare, these sort of issues could drive you insane, because they are hard to reproduce. By getting the list and the related records all in one go, that problem is completely avoided.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.