Java REST services - What are best practices regarding DTOs?

Question

I've been working on a project that communicates with a service which sends very complicated REST responses. Current "best practice" on this team is that we use POJO DTOs to handle all data transferring. If we have a GET endpoint, it returns a POJO DTO, if it's a PUT endpoint, we only accept data that fits a DTO etc. This works great for simple services, but has not worked great when we're expecting incredibly complicated JSON objects being sent to us. The DTO package is currently 15 classes large and it's not done yet; and this is only for one endpoint. One specific issue I've come across is that there are "flavors" of JSON objects that we have. Take for example:

{
  "Object" : {
    "ID" : int,
    "some other value" : int,
    "type" : "ObjectA",
    "largeDataArray" : [
      //other nested stuff
    ]
  }
}

This skeleton object may have three or four "flavors" that are all the same except for the largeDataArray that is wildly different between cases. In order to deal with this we're using jackson inheritance, but this is adding a lot of complexity to the project and doesn't work for other instances where the type field doesn't match up with the variety of largeDataArray we receive. Using jackson inheritance also has us having a @RequestBody annotation to points to the parent of a JsonType which is extended by the child POJO DTOs that we're actually working with.

This has all left me wondering if there is a better way of handling these complicated JSON objects. There was a time where everyone just used JsonObject, but that lead to a lot of issues with readability and maintainability (people not commenting their code or poor documentation). The use of POJO DTOs has stuck because it is self-documenting. Is there a better way of handling this?

One proposal I have is to simply create one large proto-type DTO that allows us to fill in everything we want and leave everything we don't want blank, but this has gotten pushback because our OpenAPI is not then able to show that only one type of largeDataArray object can be passed per call. This is certainly the simplest solution though.

Answer 1

I don't know that there's an authoritative answer, but I think we can probably turn up a few sources that will at least imply that you are trying to solve the wrong problem.

As a warmup, I'd suggest a review of Parse, Don't Validate by Alexis King

a parser is just a function that consumes less-structured input and produces more-structured output.

Given that you are starting from an HTTP message-body, which is essentially a sequence of bytes, and trying to arrive at an in memory representation that's actually useful, that seems like a pretty good fit.

Where does your parser come from? The "best practice" answer is that you generate it from a copy of the schema. See, for example, Michael Bryzek's 2018 talk Designing Micro Service Architectures the Right Way. Since schema automation isn't where your competitive advantage lies, you should be looking at what is available off the shelf, something analogous to, for example OpenApi Generator.