5

I am looking for an explanation on just why XML needs to be validated. I have been testing DTD's and Schema's for the past month and recently tried to find out why I am doing this. I know it makes the XML better and more Semantic but what are the overall benefits of validating XML?

  • 13
    That's basically the same question as "why have correct, useful data instead of random garbage?" – Michael Borgwardt Apr 13 '13 at 12:12
  • I hope this doesn't mean you aren't validating your HTML. – Rob Apr 13 '13 at 12:24
  • I do validate my HTML, I try hard to anyway! :) – deucalion0 Apr 13 '13 at 12:32
12

To make sure that it works. Sure, if you only write out some data for your own application to read it may be enough if it just works. If you send a file to somebody else matters may be different. But even within your application you may later choose to switch the parsing library and the new one may complain about errors the old one accepted and ignored like not properly escaping some characters.

Or, depending on the kind of data you store, some new characters may appear (maybe a user with a foreign name) and from that moment proper character set definitions may be important. Same if you just send an XML file from Windows to a Linux system. The closer you keep to definitions and validations the less likely you run in unexpected trouble when things change.

  • Thanks Thorsten that makes a lot of sense, its good to read actual reasons on why its important to validate XML! :) – deucalion0 Apr 13 '13 at 12:14
  • 1
    @deucalion0, you must have seen a few things crash into your face to value the small details :) – thorsten müller Apr 13 '13 at 12:15
  • Not really because I never used the XML file until I had a Schema which was correct. I created a web application that uses XSLT to provide content to the XHTML pages from the XML file as a data source. I suppose I should have experimented but at that point I just took for granted it had to be valid, end of story! – deucalion0 Apr 13 '13 at 12:18
8

It depends on how you're using the XML.

For example if you wish to pick out one or two pieces of data out the XML then move on then it would be best just to extract them and move on.

However if you need the whole XML document, so for example you want to transform it to some other XML format that needs all the fields, then I would get a schema involved.

It also depends on the relationship you have with the person sending you the XML : if you're receiving XML from a party you need to be formal with (e.g. another business) then having XML schemas allows you to establish a contract about what you will accept as input and give as output.

On the other hand if you have an informal relationship with then you can be generous with what you will accept / reject.

6

It's a bit like asking, why do we test our programs. We do it in case they are wrong.

Of course there are scenarios where you know the data isn't wrong, because the program that generates it is well tested and well trusted. In that case you don't need to validate the data. It's not compulsory.

1

Well, you don't need to. Validating XML against a schema (DTD is outdated garbage) is equivalent to using a statically typed language. It ensures a higher level of correctness at the expense of more upfront labor. I generally find that validating XML is very useful when you're dealing with remote services. It's self-documenting. If you're just transferring data from a client to server where you control both sides, then it's less useful.

1

Speaking in terms of using XML for web services: A well-formed schema lets the consumer of the web service know what the server expects and then what the server will give you back. It really helps knowing what the inputs and outputs for the web service are.

Then there are the badly-formed web services: they tell you what they expect from you, but then return some nonsense. I am currently dealing with such a service: they tell you how to form the input, but then they return the response as a string. A string containing complex XML. Said complex XML has no schema to validate against, so I have no idea what I am guaranteed to get back from the server, what is optional, or what types are possibly returned. It is effectively useless, as it tells me nothing useful.

-1

Why? because you don't want to write validators yourself, and if you do, it is very likely you aren't experienced in doing so, and when you need to change it to meet new requirements, you'll have an awful time.

Despite common belief, XML is not rigid; as long as your XML is syntactically correct, it is valid XML, despite the ignorant people telling you otherwise.

You WANT to validate your XML if your applications(be it web or otherwise) rely on models(not necessarily XML, because any model can be transformed to XML by writing a simple tree walking program) to behave correctly.

For example, you might have a JSON model for colors

[
    {"name": "red", "cssxrgb": "#ff0000"},
    {"name": "blue", "cssxrgb": "#00ff00"},
    {"name": "green", "cssxrgb": "#0000ff"}
]

Most programmers, if asked to write a program to make sure that all entries in this array are valid, and say what isn't valid when it isn't, will write something of this smell:

var jsonString = `
[
    {"name": "red", "cssxrgb": "#ff0000"},
    {"name": "blue", "cssxrgb": "#00ff00"},
    {},
    {"name": "green", "cssxrgb": "#0000ff"}
]
`;
var data = JSON.parse(jsonString);

for (var i = 0; i < data.length; ++i) {
    var row = data[i];

    if (!row.hasOwnProperty('name')) {
        throw new TypeError('row #' + i + " is missing required property name");
    }
    if (!row.hasOwnProperty('cssxrgb')) {
        throw new TypeError('row #' + i + " is missing required property cssxrgb");
    }
}

This can be difficult to maintain, requires very careful coding, and is very fragile to typos, field changes, so on.

A simple program can be written to transform the above to XML, and most models already have many online transformers from them to XML. It doesn't even have to be a good transformer, It's easier to change the method of transforming data to xml than changing custom validators. Here's an example for transforming above to

<colors><color name="red" cssxrgb="#ff0000" /><color name="blue" cssxrgb="#00ff00" /><color name="green" cssxrgb="#0000ff" /></colors>

transformation code:

var jsonString = `
[
    {"name": "red", "cssxrgb": "#ff0000"},
    {"name": "blue", "cssxrgb": "#00ff00"},
    {"name": "green", "cssxrgb": "#0000ff"}
]
`;

var data = JSON.parse(jsonString);
var xmlString = '<colors>';

for (var i = 0; i < data.length; ++i) {
    var row = data[i];
    var name = row.name;
    var cssxrgb = row.cssxrgb;
    var keys = Object.keys(row);
    xmlString += '<color ';    
    var tmp = [];
    for (var j = 0; j < keys.length; ++j) {      
        var key = keys[j];
        var value = row[key];
        tmp.push(`${key}="${value}"`);
    }    
    xmlString += tmp.join(', ') + ' />';    
}
xmlString += '</colors>';
console.log(xmlString);

Now to validate, all you have to do is run it against the doctype such as

<!ELEMENT colors (color*)>
<!ELEMENT color EMPTY>
<!ATTLIST color name CDATA #REQUIRED>
<!ATTLIST color cssxrgb CDATA #REQUIRED>

You can check it with online validators as such

<!DOCTYPE colors [
    <!ELEMENT colors (color*)>
    <!ELEMENT color EMPTY>
    <!ATTLIST color name CDATA #REQUIRED>
    <!ATTLIST color cssxrgb CDATA #REQUIRED>
]>
<colors><color name="red" cssxrgb="#ff0000" /><color name="blue" cssxrgb="#00ff00" /><color name="green" cssxrgb="#0000ff" /></colors>

The strict validators will reject the model if it contains any attributes/elements that aren't declared in the DTD, which is useful for when your programs are fragile to key,value pairs that it doesn't expect. The problem is if a validator stops validating after first error, or if there are too many false positives like undeclared tags than violations of the things your doctype actually asks to check for.

If you do know how to write your own validators, you can often write better ones faster than writing your own model-to-xml transformers and your own doctype, but even in this case, it's easier to change an existing DTD or model to xml transformer than having to modify your own custom validator.

As for HTML/SVG/other common formats; They are usually invalid XML, but if you do choose to make them valid XML, you get the ability to add your own tags which are ignored by the document if you style them to display: none, but still have feedback when you accidentally made a typo in your own model. This is useful for things like angular templates, custom metadata that gets handled by JavaScript when certain events trigger.

In the very worst, it lets you have a way to tell which parts of DOM are "standard", and which are either metadata(eg author tag in head)/custom(eg gallery tag that angularjs will transform later)/junk data(typos, metadata that's no longer needed, so on), in each case, those custom tags introduce complexity to your model, and validating xml can help you be aware of which parts of DOM might be causing the most trouble.

To summarize here are the benefits of validation

  1. it is validatable - even if it doesn't use validation, you can at least know that it is possible to validate because it has a syntactically correct XML structure, allowing you to validate it when you want to or when need comes.

  2. It helps find parts of DOM that a program depending on the model might not expect.

  3. It is simpler to write and maintain a simple validator by transforming to XML and writing a DOCTYPE and pick and choose which DOCTYPE to use on a specific validation run than writing your own validator. It is also simpler to fix edge cases in your custom transformer, doctype than inside a custom validator.

  4. It can exploit XML technologies - if it's valid XML, it can use many of the XML technologies such as XSL easily. These technologies usually add a lot of complexity over doing things conventionally, but they do allow pretty powerful ways to control your document once you get comfortable with any of them.

  • This whole thing blows up if you name a color \">red. – Polygnome Jun 19 '18 at 17:44
  • This edge case is there intentionally to showcase that it is much easier it is to change the transformer to handle this case(there are many sanitizers and you can just look for the culprit characters) than having to modify a custom validator. – Dmitry Jun 19 '18 at 17:56
  • You are validating the wrong thing, tho. Its perfectly valid data in JSON, and without any further specification, it should be recognized as valid. You have just moved the bugs from the validator into the transformer. And you are no longer testing whether your data is valid, but whether the transformation of your data is valid. Those are different things. – Polygnome Jun 19 '18 at 18:13
  • not quite, valid syntactically and valid model is different. While the syntax is valid JSON, JSON itself can't validate whether it is missing certain fields that are required, and it can be tricky expressing which ones are required, which ones are optional, which ones are unexpected. XML offers a generic solution to this via DTD. – Dmitry Jun 19 '18 at 19:50
  • Thats only helpful if your requirements are already in that form. If you never need XML, using it for validation just introduces another source of bugs - as I have demonstrated with your code. I mean XML validation is tremendously powerful and helpful, but not if used just for the sake of using it. Model consistency can be enforced in dozens of other ways, all equally powerful. You might not even be able to transform your model sensibly into XML. And the question was "why validate XML"?, not "For what other purposes can XML validation be exploited". – Polygnome Jun 19 '18 at 21:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.