Is it a bad practice to query remote resources inside validation?

Question

Watching this talk about Symfony Forms and validation, at 23:06, I got stuck with a doubt I couldn't find a satisfying answer to:

Is it a bad practice to execute a query (or a web request, or a side effect in general) inside a validator?

In my case, I have either Symfony Validation Constraints and some implementation of the Specification Pattern.

I am wondering how bad it is to perform queries to the database into these validation elements, given the fact that they could be used multiple times in the same controller (or in the same form, in the case of using Symfony components).

• there's a performance concern; especially in case the request into the validation is always the same and there's no real reason to send it multiple times.
• there's a failure concern; what if the response is delayed or, even worse, the request fails?
  • should the validation add an incident/violation because it wasn't able to ensure that the data is actually valid (even in the case where it actually is) and report it together with the other incidents/violations ?
  • should it accept it instead because of the impossibility to validate?
  • should it try again?
  • should it throw an error?
• there's a boundaries crossing and lack of separation of concerns; I would not expect that anything from the Domain Model (IMHO, including its invariants/validation-rules) knows about or has dependencies on anything from the Infrastructure (the persistence layer in this case).

Is it acceptable? Should it be avoided? Is it totally fine? I generally have a feeling of rejection in doing that (and that's why I am asking).

Answer 1

It should be strongly avoided (for the reasons you mention), but is sometimes better than the alternatives. Where necessary, it's really common to make a (local) cache to mitigate the performance concerns and to decrease the failure concerns.

Answer 2

Yes.

The reason is simple. It is always possible for the caller (service) of the validation component (model) to issue a query for the appropriate data and pass it as a parameter instead of having the validation component (model) send the query itself.

For example:

public function validateData($database, $dataId)
{
    $data = $database->getData($dataId);
    // do validation
}

can be refactored to:

// caller is responsible for getting data

public function validateData($data)
{
    // do validation
}

The upside of this (aside from the obvious testing implications) is that, because it pushes data retrieval as "high" as possible in your application, it allows for better optimization of querying/caching. Say the caller intends to validate two pieces of data. It may be possible to issue a single query, parse the response into the appropriate slices of data, and pass each to the corresponding validation component.

Answer 3

I agreed. I have the same feeling of rejection when it comes to calling external services. How @Telastyn say, some times is necessary so worth doing a bit of research to meet in deep the tools and the alternatives at hand.

@Telastyn also mentioned caches. In line with his argument, I suggest meeting your tools first because some of them implement cache out of the box. It's the case of DB drivers, WWW architecture, ERM, frameworks, etc.

For example

given the fact that they could be used multiple times

Research whether your DB, driver, ERM or framework is already caching statements. If they do, many of the validation queries will be cached eventually. For a while. The same applies to HTTP requests (if you set the proper headers).

However, working with caches can be painful, overall if we don't understand how, when and what is invalidated.

Regarding performance, often we put the focus on it too soon. Start implementing first the easiest solution, perform load tests and get real metrics. Then you will be in a better position to make decisions because these will be ground and backed by data.

To finish, when it comes to communication failures, worth knowing patterns and strategies. For example Circuit Breaker, Retry or Fail Fast. These won't save you from the Fallacies of the Distributed Computing, but they can keep the actual solution simple and reliable.

Summarising, if you can avoid it, then do it. If you can't, start with meeting your resources at hand, implement something simple and test it. Then look for enhancements or alternatives if need it.

Answer 4

You should have code that queries some resource, and eventually either returns a validated result, or some error. Usually this will be asynchronous.

If it is asynchronous then it is no problem at all to do the query, and then the validation does another query. For the user of the code, it makes no difference.

The problem is that you have twice the latency. So if you know that the resource you need for validation is always needed, and you know which resource is needed, then it would be more efficient but slightly more complicated to do both queries simultaneously, and produce the validated result when both resources arrive.

There may be situations where you have no choice. Let's say you query a customer number, and then for the validation you must check whether the customer doesn't owe you more than £1,000. Since you don't know the customer number ahead, the validation must wait to query the information about the customer until the customer number is known. It's not nice, but you do what you need to do.

PS. No need to be afraid that a query needed for validation could fail. You will be making two requests - for the data, and for things you need to validate it. Each could fail, but it's unlikely. Just report each failure as a failure. Don't say "here's the data, but I couldn't verify it", say "your request for validated data failed". (And usually failures are not random, but due to some problem somewhere along the line. If your first query succeeded, then you know your internet connection is fine, the DNS server is alive, and the server hasn't crashed, so the query for validation is much more likely to succeed).