I am reading "The Pragmatic Programmer" . The book has the following paragraph

Dave once worked on an international telex switch. Quite understandably, the client demanded an exhaustive test specification and required that the software pass all tests on each delivery. To ensure that the tests accurately reflected the specification, the team generated them programmatically from the document itself. When the client amended their specification, the test suite changed automatically. Once the team convinced the client that the procedure was sound, generating acceptance tests typically took only a few seconds.

I am curious to know how he created tests from specification . What format was the specification in ? How to write a program that will automatically read the specification and write test cases. Have I misunderstood the above paragraph ?

  • Although I don't know whether they used FitNesse or something else, it is worth having a look at. May 18, 2011 at 9:29

2 Answers 2


What format was the specification in ?

Not informal English, you can be sure.

There are a lot of more formal specification languages.

Start here: http://en.wikipedia.org/wiki/Specification_language


Z is popular for this kind of thing.

To an extent, one can use UML for this. Although it's rarely rich enough to lead to unit tests. And if the UML picture is rich enough to lead to unit tests, it can also lead directly to working code.

However, I've been very successful using spreadsheets. Yes. Plain old spreadsheets.

Most folks who use ordinary business software can prepare a spreadsheet of examples.

How to write a program that will automatically read the specification and write test cases.

  1. Parse the formal specification language.

  2. Emit unit tests in your favorite programming language based on the specification.

This isn't as easy as it seems because it still requires considerable manual effort to work out detailed examples of the edge and corner cases.

Total automation from specification to test cases with no manual intervention is only possible when the changes are relatively minor or the test data is trivial to generate.

Phone switches are (just) state machines, so it's not difficult to work out example cases.

For accounting software, however, (or actuarial or financial modeling or econometrics, stuff I've been doing recently) it's a bit dicey to find examples "automatically". It usually requires a spreadsheet showing the edge case and how it should be handled.


Given the state of natural language processing, this can only have worked if the document was already in a nearly-machine-readable state. It may have listed the expected output signals for every desired combination of inputs (like the specification for an electric circuit) or given a similar exhaustive description in repetitive language. (Perhaps it was even automatically generated itself!) In general, you cannot transform a spec into tests or code automatically. That's why programming is still necessary, after all.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.