Common test cases for XML schema validator

Question

I just wrote my first SAX-based (Java) XML schema validator, where you pass it a schema URL (pointing to the .xsd file), and an XML instance, and it determines whether or not that instance is valid or not.

I'm now looking to write several unit tests for the validator (TDD people, I know, I know!) and was wondering what were some common reasons why XML fails validation.

There are so many elements and attributes in this schema, I could literally write thousands of test cases. I figured someone here might know of a way to simplify them all down to a manageable set.

Answer 1

You might want to come up with more generic types of errors, rather than element-specific errors (unless you have specific elements that actually need that kind of testing). Some of the ones that I would recommend, just off the top of my head:

• Elements in the incorrect order.
• Missing mandatory elements.
• Invalid lengths of strings on elements that have a length limit.
• Invalid element value for an enum-type.
• Add children when the parent has xsi:nil="true".
• Have no children when child elements are required.

These are the types of validation errors I encounter most frequently on my current project. They can happen to almost any element, so it doesn't matter if the schema has dozens of elements or thousands.

Answer 2

You see that there are many tests waiting to be written - very good! Bring order into your ideas of possible tests and get a plan like this:

Go through every tag/attribute and think about things which could go wrong (mostly no subtag/attribute when there should be one or the other way round) and how it should work (often there are only two or three ways which make sense for your specific xsd). Each of these cases should be become a particular test.

Then it's very good to think about other conditions (these are very dependent on your xml and what for you need/use it):

• If you have names: should they be unique (maybe in groups or over all)
• Are all used names defined and vice versa (second is maybe not so important - but a good warning is gold)
• Other combinitions of "if you have this tag/attribute you need another one too at another place!"
• Maybe like a certain order in your xml (at first the basic and then combined subtags of a certain kind).