32

Data input validation always was quite an internal struggle to me.

On the verge of adding a real security framework and code to our legacy application rewrite project (which so far pretty much keeps the card-castle-strong legacy security code and data validation), I'm wondering again about how much should I validate, where, etc.

Over my 5 years as professional Java developer, I created and refined my personal rules for data input validation and security measures. As I like to improve my methods, I'd like some hear some ideas from you guys. General rules and procedures are fine, and Java-specific ones too.

Summarized, these are my guidelines (exposed on a 3-tier web application style), with brief explanations:

  • 1st tier client side (browser): minimal validation, only invariable rules (mandatory email field, must select one item, and the like); use of additional validation like "between 6 and 20 characters" less frequent, as this increases maintenance work on changes (may be added once the business code is stable);

  • 1st tier server side (web communication handling, "controllers"): I don't have a rule for this one, but I believe only data manipulation and assembly/parsing errors should be handled here (birthday field is not a valid date); adding further validation here easily makes it a really boring process;

  • 2nd tier (business layer): rock solid validation, no less; input data format, ranges, values, internal state checking if method cannot be called anytime, user roles/permissions, and so on; use as little user input data as possible, retrieve it again from database if needed; if we consider retrieved database data as input too, I would only validate it if some specific data is known to be unreliable or corrupted enough on the DB - strong typing does most of the job here, IMHO;

  • 3rd tier (data layer/DAL/DAO): never believed much validation is needed here, as only the business layer is supposed to access the data (validate maybe on some case like "param2 must not be null if param1 is true"); notice however, that when I mean "here" I mean "code that access the database" or "SQL-executing methods", the database itself is completely the opposite;

  • the database (data model): needs to be as well thought, strong and self-enforcing as to avoid incorrect and corrupt data on the DB as much as possible, with good primary keys, foreign keys, constraints, data type/length/size/precision and so on - I'm leaving triggers out of this, as they have their own private discussion.

I know early data validation is nice and performance-wise, but repeated data validation is a boring process, and I admit that data validation itself is quite annoying. That's why so many coders skip it or do it halfway. Also, every duplicated validation is a possible bug if they are not synchronized all the times. Those are the main reasons that I'm nowadays preferring let most of the validations up to the business layer, at the expense of time, bandwidth and CPU, exceptions handled on a case-by-case basis.

So, what do you think about this? Opposite opinions? Do you have other procedures? A reference to such topic? Any contribution is valid.

Note: if you are thinking the Java way of doing things, our app is Spring based with Spring MVC and MyBatis (performance and bad database model rule out ORM solutions); I plan to add Spring Security as our security provider plus JSR 303 (Hibernate Validator?)

Thanks!


Edit: some extra clarification on the 3rd layer.

3
  • My advice is to study how Hibernate Validator works. I haven't found JSR 303 to be useful, for the validation kicks in during persistence, whereas some of my rules had to be enforced much before persistence, for I had business rules that relied on basic validation. In my opinion, It works for a very closeted model; maybe I was using it incorrectly, but I never found anyone with experiences differing from mine. Jun 2, 2011 at 15:54
  • @Vineet Reynolds I already used it for form validation with Spring MVC, it is really great combination. I get server side validation with fine-grained messages with little to no effort, appropriate error displayed to the user. I am still to test it entirely on server side objects, not sure of the advantages. Take a look at this sample post, that's just how I used it: codemunchies.com/2010/07/…
    – mdrg
    Jun 2, 2011 at 17:00
  • 2
    put too much validation. EveryWhere damn those user inputs @#! !@@!
    – Chani
    Jun 3, 2011 at 7:40

5 Answers 5

18

Your validation should be consistent. So if a user enters some data on the web form which is determined to be valid it shouldn't be rejected by the database layer because of some criteria you didn't implement on the client side.

As a user nothing would be more annoying that entering a pageful of data apparently correctly only to be told after a significant round trip to the database that something was wrong. This would be particularly true if I'd tripped some client validation in the process.

You need to have the validation at various the various levels as you're exposing these and potentially have no control over who's calling them. So you need to arrange (as far as possible) for your validation to be defined in one place and called from where ever it's needed. How this is arranged will depend on your language and framework. In Silverlight (for example) you can define it on the server side and with suitable attributes it will get copied to the client side for use there.

3
  • 2
    +1 Absolutely. I was going to say the same thing about ASP.NET MVC, but you beat me to it. :) Really, we only NEED validation in on place in order to make sure a system stays in a valid state. The rest of the validation like client side are to enhance usability and time wasted for the user, so that should be the main focus. Consistency is key.
    – Ryan Hayes
    Jun 2, 2011 at 15:16
  • 3
    About the "round trip", I see no problem as long as the page gets reloaded with proper error messages and all fields filled out with whatever you typed before (most interfaces fall short in this last detail). If it takes too long to get back with the errors, then that's a candidate for additional client side validation.
    – mdrg
    Jun 2, 2011 at 16:47
  • And sure, if validation can be replicated easily across the app, there's no reason to waste that. On the server side it is easy, but on the client side, without such validation tools such as the one you mentioned, it gets very frustrating (ie: writing a lot of JS validation code, just like the one you wrote at the server).
    – mdrg
    Jun 2, 2011 at 16:50
11

•3rd tier (data layer/DAL/DAO): never believed much validation is needed here, as only the business layer is supposed to access the data (validate maybe on some case like "param2 must not be null if param1 is true").

This is so wrong. The most important place to have the validation is in the database itself. The data is almost always affected by more than the application (even when you think it won't be) and it is irresponsible at best to not put proper controls into the database. There is more loss of data integrity from a decision not to do this than any other factor. Data integrity is critical to the long term use of the database. I have never seen any database that failed to enforce integrity rules at the database level that contained good data (and I have seen the data in literally thousands of databases).

He says it better than I do: http://softarch.97things.oreilly.com/wiki/index.php/Database_as_a_Fortress

2
  • I agree to the very last bit with this article, I guess I haven't made myself clear at this part. I updated the question with further details. Thanks!
    – mdrg
    Jun 2, 2011 at 16:31
  • The problem with "validation in the database itself" is the app code has lost control over what is valid and what is not. Moreso, if multiple apps want to use the system then they all have to conform to the same rules and changes would be a duplicate DB, instead of just writing different code based rules. DB just stores data and what that data can be (string, int, not null if critical) db shouldn't constrict "how" the data can be used.
    – James
    Sep 1 at 3:10
11

In a relational system, I see it as a three-layered approach. Each layer is constrained by the ones below:

  • Presentation/UI
    • simple input validation
    • do not proceed if the input is in the wrong format
    • "gate" client requests to the server to reduce round-trips, for better usability and reduced bandwidth/time
  • Logic
    • business logic and authorization
    • don't let users do things they aren't allowed to do
    • handle "derived" properties and state here (things that would be denormalized in the database)
  • Data
    • the essential data integrity layer
    • absolutely refuse to store any junk
    • the DB itself enforces data formats (int, date, etc.)
    • use database constraints to ensure proper relationships

The ideal answer to this would be a system that lets you define the constraints at all three layers in one place. This would involve some code generation for SQL, and at least some data-driven validation for the client and server.

I don't know if there's any silver bullet here... but since you're on the JVM I'd suggest looking at Rhino to at least share JavaScript validation code between the client and server. Don't write your input validation twice.

1
  • I'll take a look at Rhino. If it can integrate somehow with Spring MVC form validation, so much better.
    – mdrg
    Jun 2, 2011 at 17:04
2

Two short, general rules for validation:

If you're going to call anything that doesn't guarantee it will return something (error, exception) to tell you about invalid input in a way that you can pass back to your caller, validate it.

If you're going to do anything else with the data (make decisions, do math, store it, etc.), validate it.

2

All the above make the assumption that developers and maintainers are perfect and write perfect code that always runs perfectly. The future software releases know about all the assumptions you made and never documented, and users and hackers who put data into the system in ways you never imagined.

Sure, too much validation is a bad thing, but assuming programs, networks and OS's are perfect, hackers won't get through your firewall, an DBAs won't manually "tweak" the database is probably worse.

Draw boundary circles around things, identify the failure modes it's protecting against and implement an appropriate level of checking for that boundary. For instance, your database should never see invalid data, but how could it happen and what if it does? Who is your user, whats the cost of failure?

Study physical world security models, security should be in layers, like an onion. One thick wall is considered poor security. Data validation should be considered the same way.

Not the answer you're looking for? Browse other questions tagged or ask your own question.