Data input validation - Where? How much? [closed]

Question

Data input validation always was quite an internal struggle to me.

On the verge of adding a real security framework and code to our legacy application rewrite project (which so far pretty much keeps the card-castle-strong legacy security code and data validation), I'm wondering again about how much should I validate, where, etc.

Over my 5 years as professional Java developer, I created and refined my personal rules for data input validation and security measures. As I like to improve my methods, I'd like some hear some ideas from you guys. General rules and procedures are fine, and Java-specific ones too.

Summarized, these are my guidelines (exposed on a 3-tier web application style), with brief explanations:

• 1st tier client side (browser): minimal validation, only invariable rules (mandatory email field, must select one item, and the like); use of additional validation like "between 6 and 20 characters" less frequent, as this increases maintenance work on changes (may be added once the business code is stable);

• 1st tier server side (web communication handling, "controllers"): I don't have a rule for this one, but I believe only data manipulation and assembly/parsing errors should be handled here (birthday field is not a valid date); adding further validation here easily makes it a really boring process;

• 2nd tier (business layer): rock solid validation, no less; input data format, ranges, values, internal state checking if method cannot be called anytime, user roles/permissions, and so on; use as little user input data as possible, retrieve it again from database if needed; if we consider retrieved database data as input too, I would only validate it if some specific data is known to be unreliable or corrupted enough on the DB - strong typing does most of the job here, IMHO;

• 3rd tier (data layer/DAL/DAO): never believed much validation is needed here, as only the business layer is supposed to access the data (validate maybe on some case like "param2 must not be null if param1 is true"); notice however, that when I mean "here" I mean "code that access the database" or "SQL-executing methods", the database itself is completely the opposite;

• the database (data model): needs to be as well thought, strong and self-enforcing as to avoid incorrect and corrupt data on the DB as much as possible, with good primary keys, foreign keys, constraints, data type/length/size/precision and so on - I'm leaving triggers out of this, as they have their own private discussion.

I know early data validation is nice and performance-wise, but repeated data validation is a boring process, and I admit that data validation itself is quite annoying. That's why so many coders skip it or do it halfway. Also, every duplicated validation is a possible bug if they are not synchronized all the times. Those are the main reasons that I'm nowadays preferring let most of the validations up to the business layer, at the expense of time, bandwidth and CPU, exceptions handled on a case-by-case basis.

So, what do you think about this? Opposite opinions? Do you have other procedures? A reference to such topic? Any contribution is valid.

Note: if you are thinking the Java way of doing things, our app is Spring based with Spring MVC and MyBatis (performance and bad database model rule out ORM solutions); I plan to add Spring Security as our security provider plus JSR 303 (Hibernate Validator?)

Thanks!

---

Edit: some extra clarification on the 3rd layer.

Answer 1

Your validation should be consistent. So if a user enters some data on the web form which is determined to be valid it shouldn't be rejected by the database layer because of some criteria you didn't implement on the client side.

As a user nothing would be more annoying that entering a pageful of data apparently correctly only to be told after a significant round trip to the database that something was wrong. This would be particularly true if I'd tripped some client validation in the process.

You need to have the validation at various the various levels as you're exposing these and potentially have no control over who's calling them. So you need to arrange (as far as possible) for your validation to be defined in one place and called from where ever it's needed. How this is arranged will depend on your language and framework. In Silverlight (for example) you can define it on the server side and with suitable attributes it will get copied to the client side for use there.

Answer 2

•3rd tier (data layer/DAL/DAO): never believed much validation is needed here, as only the business layer is supposed to access the data (validate maybe on some case like "param2 must not be null if param1 is true").

This is so wrong. The most important place to have the validation is in the database itself. The data is almost always affected by more than the application (even when you think it won't be) and it is irresponsible at best to not put proper controls into the database. There is more loss of data integrity from a decision not to do this than any other factor. Data integrity is critical to the long term use of the database. I have never seen any database that failed to enforce integrity rules at the database level that contained good data (and I have seen the data in literally thousands of databases).

He says it better than I do: http://softarch.97things.oreilly.com/wiki/index.php/Database_as_a_Fortress

Answer 3

In a relational system, I see it as a three-layered approach. Each layer is constrained by the ones below:

• Presentation/UI
  • simple input validation
  • do not proceed if the input is in the wrong format
  • "gate" client requests to the server to reduce round-trips, for better usability and reduced bandwidth/time
• Logic
  • business logic and authorization
  • don't let users do things they aren't allowed to do
  • handle "derived" properties and state here (things that would be denormalized in the database)
• Data
  • the essential data integrity layer
  • absolutely refuse to store any junk
  • the DB itself enforces data formats (int, date, etc.)
  • use database constraints to ensure proper relationships

The ideal answer to this would be a system that lets you define the constraints at all three layers in one place. This would involve some code generation for SQL, and at least some data-driven validation for the client and server.

I don't know if there's any silver bullet here... but since you're on the JVM I'd suggest looking at Rhino to at least share JavaScript validation code between the client and server. Don't write your input validation twice.

Answer 4

Two short, general rules for validation:

If you're going to call anything that doesn't guarantee it will return something (error, exception) to tell you about invalid input in a way that you can pass back to your caller, validate it.

If you're going to do anything else with the data (make decisions, do math, store it, etc.), validate it.

Answer 5

All the above make the assumption that developers and maintainers are perfect and write perfect code that always runs perfectly. The future software releases know about all the assumptions you made and never documented, and users and hackers who put data into the system in ways you never imagined.

Sure, too much validation is a bad thing, but assuming programs, networks and OS's are perfect, hackers won't get through your firewall, an DBAs won't manually "tweak" the database is probably worse.

Draw boundary circles around things, identify the failure modes it's protecting against and implement an appropriate level of checking for that boundary. For instance, your database should never see invalid data, but how could it happen and what if it does? Who is your user, whats the cost of failure?

Study physical world security models, security should be in layers, like an onion. One thick wall is considered poor security. Data validation should be considered the same way.