How to maintain consistency when retrieving partial vs. full data in an API Resource

Question

I'm working on a API for the logistics department, and I have a resource called logisticTransport, which is an entity in our database. I'm facing a challenge with maintaining consistency when retrieving data from this resource.

In some parts of our application, we only need to retrieve certain fields from logisticTransport, so we return a subset of the fields. However, in other cases, we need to retrieve the full entity, including related fields from associated entities.

This approach is causing some inconsistency because logisticTransport sometimes contains only basic fields, and other times it includes additional related fields. It has also become challenging to synchronize these different types of responses in our codebase.

For example, I have an interface in TypeScript called LogisticTransport, which represents the type expected by the frontend. This interface includes the basic fields. Should I create a separate interface when I want to retrieve the full LogisticTransport resource with all the related fields? Or would it be better to use a single interface that includes all possible data, even if some fields are not always returned?

How can we structure our API to ensure consistency across different use cases while allowing flexibility in what data is retrieved? Are there best practices for handling this kind of scenario?

Thanks in advance for your help!

Answer 1

There is no general rule you can apply for this scenario that works universally across all applications. The problem with adding everything to the LogisticTransport interface is that there is no clear indication whether you have the full information or the partial info. Since you have TypeScript available, I recommend two interfaces with good names. The hard part is naming these silly things and defining the scope of what they contain.

If the first API response is a subset of the full information, I recommend one interface for the partial information and a separate interface for the full information. I'll take a stab at naming things, but you can change the names to suit your needs.

• BasicLogisticTransportInfo — this would define those minimal fields returned by the partial response.

• FullLogisticTransportInfo — this interface has everything from the full response.

You want the names of these interfaces to communicate two things:

1. They contain information about logistic transports.
2. One contains partial information, and the other contains full information.

These could be two separate interfaces with no inheritance relationship provided you never need to pass the full information to a method that only requires partial information. This option is nice because the API responses for the full and partial information are free to evolve independently without requiring changes in the frontend code. The downside is that you need to cast or convert the "full" information to the "partial" information in cases where you need to pass the full info to a method that only requires partial info.

You could have FullLogisticTransportInfo inherit from BasicLogisticTransportInfo. Now you can pass a FullLogisticTransportInfo object to a method that requires a BasicLogisticTransportInfo object without casting or converting. The disadvantage here is coupling the two API responses together. A change in either API response could bleed over to the other interface causing downstream changes in other frontend components.

Choosing inheritance or two separate interfaces will largely depend on whether you believe the partial response will continue being a subset of the full response.

You could look into composition to separate things as well. Rather than two interfaces on the frontend, you can have one LogisticTransport interface. The partial information would be defined on this interface. Additional information provided by the full API response would then be decomposed into other interfaces which are referenced as properties of LogisticTransport. Unfortunately, I cannot give any more information about this option without knowing what each API returns. Composition might not make any sense from a REST standpoint, which means the API response would need to be restructured on the client rather than just being blindly passed along to the caller.

Answer 2

The easiest and best solution is to return the full object with all fields all the time.

This avoids a coupling between the Model and the UI, or whatever process you are creating and allows the same API to be used across multiple consumers.

In most cases the argument that less data will be used if you limit yourself to a "lite" object is untrue. It neglects to take into account caching and reuse of the object. For example you might have your list page, and an item detail page. You save on the list page by passing fewer fields, but then have to request the same data for the detail page. Where you could have just used the already downloaded object if you had sent full data the first time around.

The downside to this approach is that it forces you to be careful when defining you Models. You can't have Company.Employees[400].Address.Street. The Company object would be too big. You have to have Company and Employee with Company.EmployeeIds.

Answer 3

What you should implement here are API filters. Then the HTTP Clients can request specific data as they need.

Without filters and returning pre-defined data sets you are serving the Client's needs too tightly, and this can become messy with each Client's specific needs. E.g. what if next week the Client wants a 3rd option, then later another client wants some kind of hybrid between them all, and so on.

Without a filter, the API should return ALL data and the Client just use what it needs from that. The Client can implement it's own DTOs to map specific scenarios. So e.g. have the full object of all data, then some mapper to make BasicDto.

But again filters is the way to go here imo.

Answer 4

The fundamental thing to avoid is overwriting a fields good data, data that wasn't downloaded, with a null (or empty string). A null that only exists because the field wasn't downloaded.

The problem with encoding the "don't use the data in this field" signal as a null is that sometimes you want to overwrite good data with a null (or empty string) because it's time for that data to go away.

Well, that, and some fields just can't be null.

This problem can tempt you to split the interface into basic and full varieties. Which will work well right up until you change your mind about what field goes in which.

There is an alternative. Stop insisting this is a type. Treat it like a data structure. JSON is really good at showing you what field wasn't downloaded by simply not including that field. Collections are also good here.

Static types are not good at representing partial data. Data structures are. The rub is now you have to code against a data structure.

That's the most flexible solution. If you don't need that kind of flexibility and you're confident you know what basic and full will be now and forever then stick with static types.

If you insist on static types (because you hate data structures that much) and max flexibility then you're simply going to have to find somewhere to encode that "don't use the data here" flag.