I need help understanding what user stories should look like for a web testing framework we are building for our organization's QA department. We are trying to run this project using Scrum.

The product will have 3 layers:

  1. utility classes to make working with underlying web automation tool easier.
  2. a page-object API layer which will allow our testers to write tests from the perspective of what services our web pages offer, instead from the perspective of the web automation tool.
  3. a declarative layer, which will allow writing of tests in an English-type language--perhaps Cucumber-like scenarios. This layer will be mapped to the page-object API layer, so that if a test can't be performed declaratively, the test author can drop down into the page-object API layer.

The end-user of this framework is our QA group who will be writing tests either at declarative level, or if they need to, the page-object API level.

My question is what should our user stories look like? Currently we have stories that cover what the web automation should be able to do (ie. find elements on the page, wait for elements to appear), but that can't be right. Firstly, we could deliver this kind of stuff already by giving the QA group the underlying automation tool. These stories are clearly not worth demonstrating in a sprint review.

Secondly, stories should indicate business value, which in this case is not in finding elements on a page, it's in:

  • having tests based on an up-to-date, popular and effective web automation tool.
  • having readable tests
  • having the actual tests separated from the web automation tool which is driving them.

Fine, but how do I break the product up into more specific user-stories?

  • How do I ask them? They are more unfamiliar with Scrum and user-stories than I am.
    – Aaron
    Commented Feb 25, 2013 at 17:18
  • @nathanhayfield: No it can't. A user story identifies the user/group and the reason for wanting the feature. What you've offered is actually a solution constraint.
    – Aaronaught
    Commented Feb 26, 2013 at 0:42
  • @nathanhayfield: Not sure what kind of team you work on but we have millions of users and not a single one of them writes user stories for us. That's what product managers and (sometimes) marketing are for. If they know how to do their jobs then they write better user stories than the users ever could. And if they're really good at their jobs then they'll refine those stories over time with actual usability tests, a critical part of the process which (sadly) most Agile teams still don't bother with. You might have business units representing users, but that's not the same thing.
    – Aaronaught
    Commented Feb 26, 2013 at 0:59
  • @nathanhayfield: No, we're officially Scrum. I'm no particular fan of any of these methodologies, but Scrum (which this question is about) very specifically says that the product owner (representing the customers) writes the stories. It's fine if you and your team aren't inclined to follow some methodology to the letter - like I said, they're all pretty meh anyway - but you should probably familiarize yourself more with the details of these processes before pigeonholing this way.
    – Aaronaught
    Commented Feb 26, 2013 at 2:36
  • So, I have several pieces of advice given. One is to try to write stories that represent the tests that QA will write, and another that seems to say that I am trying to use the wrong methodology for this project. I have one person from an Agile consulting company telling me that a functional task is better for API development than user stories.
    – Aaron
    Commented Feb 26, 2013 at 13:42

2 Answers 2


I would suggest to slice vertically for stories, not horizontally:

As a QA group member, I can write a test using a page API that goes to a specific page, click on a button and verifies that I navigates to the right target page.

Then implement the page API layer and base layer necessary to support that, and no more. This is demonstrable and has business value: you can actually write a test and run it.

Then follow up with other vertical slices:

As a QA group member, I can write a test that goes to a specific page, fill in a text box, click on a button and verifies that I navigates to the right target page.

And incremental improvements (Cuke for instance)

As a QA group member, I can write a test using a cucumber like DSL that goes to a specific page, click on a button and verifies that I navigates to the right target page.

  • 1
    "Clicking on a [button/link/whatever]" is exactly the type of specification that the Page Object model and the Cucumber/Gherkin language aims to prevent.
    – Aaronaught
    Commented Feb 26, 2013 at 0:41
  • Well, fill in whatever your requirement or style of test is. The point is that the stories should be fully functional features (allow an actual test, even if minimal) rather than 'code the entire first layer', then 'code the entire second layer'.
    – ptyx
    Commented Feb 26, 2013 at 2:24
  • Exactly. We are building this framework to prevent this sort of thing.
    – Aaron
    Commented Feb 26, 2013 at 13:38

This is a dicey one. Scrum dictates cross-functional teams, which means that "user stories" for QA don't make any more sense than stories for developers. QA aren't users, they're part of the development process, and automated acceptance tests - assuming you're truly committed to having them - are best reflected as such.

When we first started doing this type of testing on my team, it was a spike. Then, it was rolled into the "testing" stage for every feature, and any framework-level development got charged to the individual tasks. This was intentional; we wanted to reflect to the management that the tests were no longer optional and that putting them off would just cause other features to take longer to develop.

When the tests start to get a little too rough around the edges, we parcel it off into tech debt and allocate a day or even a week to work on it.

I've had nothing but bad experiences with internal user stories. You're never quite sure when they're done because, as you're seeing for yourself, the acceptance criteria are wide open to interpretation.

If you're being forced to create user stories, what I would suggest is to create one story per service/action, for example:

As a tester, I would like to be able to automate a successful login via the web browser, so that I can write automated tests for logged-in features.

These stories will get repetitive, as stories often do when they're used inappropriately, but it's better than tests around finding elements on the page, which, as you say, is already offered by the framework. In this case, each story corresponds (roughly) to a Gherkin step or condition.

You could also parcel out all of the steps as tasks, and create a story for each entire page/service. Depends on how fast your team works. If you can model that entire page and all its services in one cycle then go for it. If you're not sure, stick to story-per-service or start a spike to improve your estimation.

  • Thanks for your input. That's what we do, our group. We create tools for other programmers and other groups in our organization. Are you saying that Scrum doesn't work well with this kind of "meta" work?
    – Aaron
    Commented Feb 26, 2013 at 3:00
  • @Aaron: I prefer to treat Scrum as a tool rather than a religion and I'm fortunate enough to work in an environment where the PMO generally agrees - just as we are primarily a .NET shop but have no issue using JavaScript or Python to solve certain problems. If you're actually building a clone of Cucumber/SpecFlow/whatever then yes, you should have a product owner define proper stories. But if you are just using something like that to expose the features of some other product then I'm not sure if it's really worth all the overhead of Scrum. Stories imply optionality which testing isn't.
    – Aaronaught
    Commented Feb 26, 2013 at 3:08
  • Ultimately I can only relate my own experience. Where I work we're willing to occasionally say that something just doesn't make sense in the context of Scrum and write it off as an interruption/tech debt/etc. It's up to you and your team to decide whether or not that applies to your situation; if not, well, I already offered what I could in terms of story-writing here.
    – Aaronaught
    Commented Feb 26, 2013 at 3:10
  • In this case it's not an interruption--it's what our group does. And, yes, there is a customer-the QA team. The product isn't a clone of cucumber--it's a tool for the QA team to write proper automated tests. I think I am going to go in the direction of writing stories that cover their tests. For example, as a QA member I need to login to x application so I can test y. Then I will assume that this tests needs to be performed at two different levels, the page API level and the declarative level. The story is complete when the user can write the test from both levels. Thanks for everyone's input.
    – Aaron
    Commented Feb 26, 2013 at 14:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.