Is there an effective way to test XSL transforms/BizTalk maps?

Question

Creating repeatable tests for BizTalk maps is frustrating. I can't find a way to handle testing them like I'd do unit testing, because I can't find ways to break them into logical chunks. They tend to be one big monolithic unit, and any change has the potential to ripple through the map and break a lot of unit tests. Even if I could break it up, creating XML test inputs is painful and error prone.

Is there any effective way of testing these? I'd settle for recommendations for testing XSL transforms in general, but I specifically mention BizTalk maps primarily for the reason that when using the mapper, there really isn't any way to break your XSLT into templates (which I'd imagine you could use to break up your logic into testable chunks, but I've honestly never gotten that far with XSLT).

Answer 1

I'm not an expert on BizTalk, but I made some unit tests for XSLT. I found several ways to do it.

1. Creating some XML that represent the usual data.
2. Developing a reverse tool (from XSLT output to significant XML data) and then testing XSLT over a big amount of XML (e.g. monthly invoicing, integration messaging, etc.). If you can obtain the significant part of the XML you are in the good way. This test can be presented to a customer to ensure that the XSLT was made correctly.

We use the way 1 to develop XSLT and the way 2 to test XSLT.

Answer 2

It depends on what you want to prove with your tests.

If you want to prove that the transform works, you can use one or more input messages, apply the transformation, and use validation on the output message.

More specific validations can be done with XPath checks on specific fields. These kind of tests can be used to validate upto 80% of the working of the transform.

If you want to test specific corner cases (what happens when a node is not present, when this node is empty, etc..), you can create specific test messages, this will help you to reach even higher confidence in the correctness of the transform.

I think that in most cases it is not cost effective to try and test every line and every possible scenario using the transform. Therefore i would focus on the easy cases first, and specific testcases only when you see a risk, or an actual defect was discovered during testing.

Most important is to have enough real test messages as input. Try to collect as many diffrent possible input messages from the source system, and wrte test code to apply the transform on all these testcases.

I'm currently building a set of transforms, and i'm testing them using (our in-house built) BizTalk Testing Framework. I have recenty added an XmlAssert class that can evaluate node values based on xPath expressions. That works really well. Executing BizTalk maps in unit tests is quite easy.

Answer 3

For testing XSLT I rely on two command-line tools: the hopefully familiar xmllint for validity and xmlstarlet for data validation. The latter allows you to create shell scripts to pull out data elements for testing within that script. You can use arbitrary XPath and so test things like the number of elements in an output document. Thus you can create a test directory next to your XSLT which has a sample input document along with a script to produce the output and test it.