4

Can anyone offer any tips or experience on developing a test plan for an Agile sprint? Most of the time our team simply creates a "Test Feature X" task and goes about writing ad-hoc test cases. There typically isn't much strategic forethought put into how features will be tested and how we'll communicate the expectations and results to the rest of the development team and our users.

Is it common to just write high-level testing tasks without making the actual test cases transparent to the development team? What sort of methodologies have you used when developing a test plan and what does it look like both at the beginning of a sprint and at the end? Any advice or tips on what comprises an effective and transparent test plan as well as a means of developing the test plan would be much appreciated.

|improve this question
  • What's the purpose of the test plan for you and your team? – dzieciou Mar 15 '13 at 17:52
  • @dzieciou Honestly all that you described in your answer below is what our ideal purpose would be. Doing everything we need to do in order to test the sprint's features and do so in a transparent manner. – KodeKreachor Mar 15 '13 at 18:28
1

I think it really depends on the type of project and what kind of decisions you must made. I.e., what is the purpose of test plan for you.

Here I will share some experience regarding our project.

In our project and team testing means:

  • preparing test environments, required for system tests
  • preparing test data: getting real data from production or simulating
  • automating tests at unit, integration, system, system integration level.
  • writing mocks, test drivers, data simulators, test frameworks
  • demonstrating development output

So planning tests is about making high-level decisions to clarify above. For instance, we decide whether QA needs to end-to-end test a given story, or it is better if she increases test coverage at unit level, extending this way tests written by developers. Getting test data often means getting in contact with customers, business analytics and other stakeholders to get access to them. End-to-end tests often require know-how of more experiences testers, business analytics and delivery guys, so we must plan ahead training. Testers often present what the team achieved, so we sometimes plan what tests to prepare for the demo.

Automation is a separate story. We have established methodology and frameworks for testing, so testing means implementing and executing test cases. On other side, we are developing a new test framework, that is a separate development project going in parallel to the main one. This must be planned as well, cannot be done ad hoc, but we still learn from other teams how do plan this effectively.

We usually know what we will be doing in the current or coming release, so planning tests is done ahead for several iterations.

We usually do some test planning with the rest of the team, when iteration starts. We do not always write down all decisions about testing, but we still plan testing process.

We, as testers, sometimes need to support both current iteration and regression tests. So we need to plan how much our capacity can be devoted to current iteration and whether developers can cover our testing tests, if we are too busy with other responsibilities. This is agile, so developers can do some testing, but we still need to plan this to some degree.

|improve this answer
  • Thanks for your excellent answer, this indeed helps! Does your entire development team create this sprint test plan, or does that responsibility go to more QA-focused team members? Also, how do you keep track of the status of all of your bullet-point items (whose working on what, what is done, what still needs to get done). Thanks again for your answer! – KodeKreachor Mar 15 '13 at 18:30
  • 1
    Some of the bullets in the plan are defined as tasks for the story in story tracking portal (usually development tasks for tests, test case ideas), often only roughly described, but sitting together with main development tasks. We track them all together. Other bullets like a need to organize a training prepare environment are discussed on the meeting and some people are assigned to take care, but we not track this. Other teams members, scrum master or our supervisor sometimes reminds about it.... Ok, sometimes we create JIRA tasks for operations to prepare environment, a way to track – dzieciou Mar 15 '13 at 21:58
  • 1
    Finally, some general test strategy (test level, risks to test against) one of us describes in a document, but we do not keep it updated. It is more useful in the beginning to define direction. This strategy we discuss rather with QA-focused team members, but we work too shortly together to say we defined a process once and forever. It evolves based on our needs. – dzieciou Mar 15 '13 at 21:58
  • 1
    Finally, one high-level test plan we defined, was only once or twice useful to someone else outside of our team: to manager or someone like that---to understand to what was tested by the team during recent few months. – dzieciou Mar 15 '13 at 22:06
2

Can anyone offer any tips or experience on developing a test plan for an Agile sprint?

First, consider where do test plans fall within your definition of a story being "done"? Is it enough to write them out at a high level? Implement them? Implement and pass them? There are various pieces here worth noting that can be debated within a team that should be a consensus to determine the standard for the time.

Is it common to just write high-level testing tasks without making the actual test cases transparent to the development team?

Could be. At various places I've worked there have been no testing done as part of development to breaking down more than a handful of tasks as part of testing as there would be unit tests, integration tests, QA having their own tests, and sign off from the end-users that could be part of our process at one place.

What sort of methodologies have you used when developing a test plan and what does it look like both at the beginning of a sprint and at the end?

TDD would be one idea you could use here. There are lots of other possibilities where the question is what are you trying to gain from having this methodology: Is it to reduce technical debt, improve quality, or something else?

The sprint demos can be a useful point to show off the functionality to users and collect feedback on what may have been missed. While this isn't technically in a "test plan," I would argue this falls under Quality Assurance which is part of the point of testing.

Any advice or tips on what comprises an effective and transparent test plan as well as a means of developing the test plan would be much appreciated.

First, document what are the issues with the current strategy you are using. Where is it failing to deliver value? Second, what are the factors you'd want in a revised test strategy? Part of Agile is about changing things and being prepared to keep adapting. If you know a structured way that the tests should be done, communicate that and see what others in the team think of such an approach. Perhaps they'll love it and want to give it a try? Perhaps they'll have questions that you didn't ponder initially?

Where I have worked where we had a number of tests there could be issues with how fragile the tests make the code because of all the dummy data we had to generate to run the tests and then if someone does change something some tests break and didn't always get cleaned up properly.

|improve this answer
1

I've found that session based testing and test automation really do shine in these kinds of cases. I'll normally start off with some high level scenarios that the stakeholders find important. From there, I start writing session charters for things that I want to test, with a little bit more detail. When I execute the session, that's when detail starts going in, recorded in the session notes.

Having test automation can also help. Even more so when using something like cucumber. Myself, using SpecFlow, I'm able to create the test in a way that my stakeholders can understand what the test is doing, and they have a high level summary of what actually passed/failed.

Although I'm not a fan of giant test plans, I understand that it's occasionally required. Instead of putting test cases into your test plan, put the session charters and test automation into it instead. Remember, it is a living document, and will change and evolve as testing continues. At the same time however, your test plan should probably be for your application/project, and not necessarily for each individual sprint.

|improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.