CRUD service with or without DTOs

Question

I'm about to create a service providing a simple CRUD Json REST-API. The main requirement is that documents stored/received always conform to a schema provided as JSON schema. So here's the thing:

Usually, I'd create DTO classes, map them to internal data classes and make sure that data written to the DB is always valid, by constraints and/or additional manual validation. But as the schema only describes API level objects, all validation would need to take place in the presentation layer. So imagine updating a certain value in the document, we'd have to load the document from the DB, map it to a DTO, perform the desired update to the DTO, check that it is still valid and then perform the actual update on the DB.

To simplify things, I could skip creating DTO classes at all and directly persist API classes in the DB. This way, I would at least not have to map back and forth and i could validate data from/to the DB directly. An even more extreme approach could be to not use data classes at all and just use maps/Json objects. As we're using a NoSQL DB, it'd not be a problem to persist them directly and validation would work as well. The service itself will only do minor things with the data, like adding an id to newly created documents, which seems manageable even without model classes.

Still, after doing DTOs for so many years and basically being happy with it, I'm still somewhat reluctant to ditch them completely.

I know there's probably not the right or wrong answer, but does someone has experience with one or the other solution and maybe has a few hints on what to take care of? Many thanks!

Answer 1

Usually, I'd create DTO classes, map them to internal data classes and make sure that data written to the DB is always valid, by constraints and/or additional manual validation. But as the schema only describes API level objects, all validation would need to take place in the presentation layer.

If the analysis makes no mention of an internal DTO or even any internal implemenation whatsoever; then the analysis inherently cannot be stating something along the lines of "the validation must happen before an internal DTO is involved".

What this means is that you have free reign over where this validation takes place, as long as the implemented validation covers the entire API as expected. You could, purely theoretically, create 5 subsequent layers and only perform the requested validation at the bottom most layer.
Not that this is the right answer here, but the point is that the analysis does not tie you to having to perform your validation in a given layer when the analysis makes no layered distinction to begin with.

To simplify things, I could skip creating DTO classes at all and directly persist API classes in the DB.

I'm not one for blanket statements. Generally speaking, I'd avoid the conflation of entities and API models.

However, when dealing with a REST API whose sole purpose is to act as a web-based data repository which should always mirror the data store's structure and has no additional business logic whatsoever; an argument can be made that the returned API model must invariably be the entities without any possible reason for them to be different; at which point the advice to separate your entity/model dissipates.

This way, I would at least not have to map back and forth

Be very careful that you're not tricking yourself into this purely because you want to avoid writing some additional API models and mappings. Avoiding the effort is not sufficient justification in and of itself. This is a very dangerous corner to paint yourself in; one which only makes sense under very strict conditions.

The service itself will only do minor things with the data, like adding an id to newly created documents, which seems manageable even without model classes.

However tiny, this is a form of business logic, and it already starts infringing on your argument that you can get away with not separating your API models from your entities.

Still, after doing DTOs for so many years and basically being happy with it, I'm still somewhat reluctant to ditch them completely.

You didn't bring this up, but it's a likely next step on your train of thought: there's little sense in having a partial abstraction whereby some entities get an API model and others don't just doesn't make sense.

You either abstract or you don't. If you go half-and-half, you're signing yourself up for the negatives from both sides (the effort of maintaining the models plus the tight coupling that hampers future changes).

I know there's probably not the right or wrong answer, but does someone has experience with one or the other solution and maybe has a few hints on what to take care of?

"I could totally cut this corner" is a dangerous proposition. It is very easy to trick yourself into it, and it can be very hard to claw your way out of it again. It is one of the biggest Great Filters that software projects run into.

Alas, there is no quick argument to defeat all of the possible reasons (or delusions) that lead you to cutting too many corners. And, from personal experience, even if you provide an apt argument that disproves the benefit of cutting a given corner, there is no guarantee that the intended corner-cutter listens to/believes your advice.

Does that mean we should never ever cut a corner? No, because that too would be a blanket statement, and I told you I don't like blanket statements.
However, whenever we think that cutting a corner might be warranted here, we need to really think about what is driving us to make that decision; and double/triple/quadruple check that there are no ill consequences that we're failing to consider at the time of making our decision.

Personally, unless I have an ironclad argument as to why there are no ill consequences, or I am immune to ill consequences (e.g. a one time use script), I avoid cutting corners where possible, because it's often safer traveling on the road instead of going through the forest.

Answer 2

As a general rule, if you can accomplish the same with less code and/or less abstractions, you should do it. I would go so far to say, you're obligated to do so.

Stated the other way: If you have a choice of doing something in a more complicated way but see no immediate benefit, then don't do that thing.

Answer 3

There appear to be a couple of misconceptions in the question:

1. all validation would need to take place in the presentation layer

   Since a REST API is open to the network, clients are free to send data to your service without validating it. While a legitimate client would only do this by mistake, a malicious client will purposefully send data that violates your business rules in order to probe the system for weaknesses or vulnerabilities.

2. To simplify things, I could skip creating DTO classes at all and directly persist API classes in the DB.

   Persisting API classes directly to the database can allow clients to specify fields in the request that you do not want modified. Consider a use case where you don't want the Id to be created or updated. If a new Id is sent in the request body, persisting the API class to the database would then persist that change in Id automatically. As long as the new value adheres to your business rules, the database accepts it. If the use case does not want the Id to be updated, then you are allowing people to bypass that rule.

   Any simplification in your architecture will then be complicated by detecting changes to properties that you don't want changed.

Consider a different perspective for a moment:

(old perspective) To simplify things, I could skip creating DTO classes at all and directly persist API classes in the DB.

(new perspective) To simplify things, I could skip creating API classes at all and directly persist DTO classes in the DB.

This changes your perspective from "API classes" to "requests from the client." Imagine all the mischief a malicious actor could wreak on your system if you take any old data from an unknown source, don't validate that data, and then blindly persist it to your database. If you aren't feeling a knot develop in your gut, you should. This is bad.

While a DTO doesn't have any logic, and you might be mapping DTO properties to entities, a DTO does serve an important purpose: it filters out parameters in the request that you do not want clients to set, modify or read. Furthermore, it gives you a layer in your architecture that safely accepts any client input so that you can validate it. Validate all inputs. Never trust the client to validate data. Malicious clients will purposefully not validate requests. You want to funnel all requests (especially malicious ones) through your business rules, no matter how simple.

Do not blindly persist requests from the network. Assume each one is malicious or faulty. Only persist client input after it has passed all of your business rule validations. Data Transfer Objects give you a safe place to receive client input before transferring that data to your persistence layer.

I am also a little confused by this sentence:

So imagine updating a certain value in the document, we'd have to load the document from the DB, map it to a DTO, perform the desired update to the DTO, check that it is still valid and then perform the actual update on the DB.

What feels over-complicated is mapping an entity to a DTO, applying an update to the DTO and re-mapping it back to the entity. That is the part you should avoid. The DTO is user input. Map the DTO to your entity after validating the DTO adheres to business rules. Remember the purpose of a DTO is to represent a serialized request from the client. Many mapping libraries will do convention based mapping across different types, which can reduce the boiler plate coding required for mapping DTOs to entities.