3

(Background - skip to bold statement for the crux)

I'm working through the book "Growing Object-Oriented Software, Guided by Tests" by Freeman & Pryce, while applying it to a project I'm starting. My app is a web app in PHP that manages a user's eBay interactions.

I'm using Behat/Mink to write the end-to-end tests. I'm stuck on the very first test.

In the book, they stress the importance of first writing tests that then guide the writing of the code to make them pass. And that the tests should not get in the way of development, including changes to requirements, refactoring, and so on.

They stress that the first test should exercise the system end-to-end; e.g. commanding the user interface, using a database, to interacting with an external system such as eBay; and that this feature should be minimal.

But this means that the GUI for the first test is unlikely to remain as-is for future tests, and therefore the first (and subsequent) tests should be as flexible as possible so that inevitable changes to the GUI won't break the tests (which would have at least two major side effects: time re-writing the tests, AND risking that those re-written tests fail to catch bugs that the originals would have caught). The latter is what really worries me - In the book "The Pragmatic Programmer" they emphasize "Find bugs once".

So that's the abstract problem. Applying it to my project, I was thinking of first writing a slice of the system that accumulates records of purchases made from the user. This would involve setting up notifications on eBay that make a call to my application each time a customer commits to buy something from the user. The functional test would simulate that, and then check via the GUI that the order appears in the list (which I'd implement initially as a simple, single HTML table) visible to the user.

But once the customer commits to buying, they then need to pay for it via Paypal. The records that the app handles then become complex, since some users may never pay at all, some will pay, some will have full refunds, partial refunds, multiple refunds, messages may be exchanged between the user and customer, the user might want to print postage labels from the interface, etc etc.

So in all likelyhood the GUI will become unrecognisable from that written to satisfy the first test feature.

How can these two problem best be dealt with?

  1. Time involved in adapting the GUI test code
  2. Risk of re-written old tests failing to catch bugs for an old feature that previous tests would have caught

I've come up with some ideas, but it feels like a pandora's box that has overwhelmed me, and I'm wondering if anyone can point me in the right direction, possibly with reference to other books etc:

  1. First idea: write the Behat tests (Gherkin) which would remain unchanged, but modify the Mink code that interpret those statements each time the GUI is changed. This solves very few problems, but at least the Gherkin statements would likely need no modifications.
  2. (Extremely vague idea) Make the GUI layer as thin as possible, possibly multiple wafer-thin layers of abstractions just for the GUI, with matching abstractions in the tests, some how in such a way that would prevent changes in the GUI from affecting much of the tests. (Honestly, I don't have a clue how this would work).
  3. Freeman/Pryce give a hint: "Make changes in the smallest possible steps". This makes it easier to correlate changes in the requirements/GUI design with changes in the tests. But ultimately as the changes in the GUI accumulate, I don't see how the risk of ineffective tests wouldn't also accumulate.

Apologies for the long question.

  • I haven't read the book, but your initial premise is wrong. Your very first test should be a unit test that tests your very first method. Software is generally "grown" from the bottom up, not the top down, and I'm surprised that the book got that wrong. The reason this is true is because almost everything that meaningfully happens in a well-architected program happens outside the UI. For the very same reasons that you articulate in your post, most genuine functionality is pushed out of the UI into another module, and this facilitates testing. – Robert Harvey Aug 3 '15 at 14:10
  • Actually, the books isn't necessarily wrong here but it's been too long since I read it to know the exact details. In the sense that you should write a failing test first, it wants you to write an integration or UI-test first and in order to make THAT pass, you write unit tests which in turn drive the writing of actual code. It also pushes you towards vertical slicing and a 'minimal viable product', instead of the traditional design where the entire database is created, next all the data access, next the domain, etc... – JDT Aug 3 '15 at 14:30
3

Don't write GUI tests to match the GUI. Write acceptance tests that match requirements

You could write your tests like this:

  • Given I enter "user" into the username field and "pass" into the password field
  • And I click the "Login" button
  • When I click the Find User button
  • And I enter "foo" into the username field
  • And I click "Search"
  • Then I should see a table with a user "Johnny Foo" in row "1"
  • And I should see a table with a user "Manny Foo" in row "2"

Or you could write them like this:

  • Given I am logged in as "user" with password "pass"
  • And I look for users called "foo"
  • Then the searchresults should show "Johnny Foo" on row "1"
  • And the searchresults should show "Manny Foo" on row "2"

The first one completely couples you to the GUI. The second one only couples you to what you need in terms of functionality, stories and requirements.

This way, you write code that becomes re-usable. The lines in your tests are functional expressions and there is code that matches these to the concrete GUI-implementation, but if something changes in your GUI you only have to make a change in one location, which is the place where you map things like "I look for users called 'foo'" to concrete GUI code.

Case-in-point: yesterday I switched a page for an application I work on from being loaded with a submit to fetching and displaying data using JSON and Knockout. That means that, instead of clicking a button and waiting for a submit I was now displaying a loading-animation which goes away when the data is loaded. Try this with the example above: in the first example you'll need to add that wait as a step, in the second you just tag that along with the "I look for users called 'foo'"-implementation. Now if you have three tests that look like my first example, no sweat. But what if you have ten? Or a hundred?

By the way, if you look at the examples for Behat/Mink and the Gherkin DSL it uses, you'll see these are very general, human-readable examples. This is not by accident, it serves as a very good example of how you should think when writing tests at this level.

  • Thanks - kind of confirms what I suspected, making the DSL reflect requirements not implementation. But seems to confirm my fears that it is near impossible to write the code that maps from the DSL in such a way that it is immune to changes in the GUI. It does make sense. I suppose these kinds of tests are not meant to thoroughly exercise anything, more just a top level indication that the features are more or less implemented and working, so there won't be that many of them. I read that acceptance tests are among the most costly, and tend to be minimal and low in quantity – CL22 Aug 3 '15 at 14:32
  • 1
    Yes, in the sense that you only test general scenarios in your application. When you acceptance-test a form that has 20 input fields, all with validation, you would unittest all the validation and write a single UI-test to check that the wiring of the validation to the UI works as intended. – JDT Aug 3 '15 at 14:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.