22

How is testing handled within the same sprint as coding, if all or most of the coding is not done until the end of the sprint? (I'm referring to the "soup-to-nuts" development and testing of a single PBI within a sprint.)

Most of the answers I've seen online involve QA automation, but even that isn't really possible since you generally need a functional UI to record or create automated tests from. I only have storyboards that continue to evolve as I develop features and discover new requirements.

In my case, I am developing a new desktop application. Desktop apps don't generally lend themselves to automated testing very well. I have some automated unit tests, but they are not the manual functional/integration tests that a QA professional would perform.

So, where I'm at now is that my sprint ends tomorrow, I still have coding to finish, and my QA folks have nothing to test yet, and no idea how to test whatever I'd give them without me holding their hands.

I'm sure I'm not the first person to have this dilemma.

In the past, I've done a pipeline: in the current sprint the test team tests the features that have been implemented during the previous sprint. At my current job, the PM refers to this approach as "waterfall", and as such, unacceptable.

  • 2
    You are not the first person to have this dilemma. You could use a pipeline: in the current sprint the test team tests the features that have been implemented during the previous sprint. – Giorgio Jun 26 '14 at 14:44
  • 2
    PM forcing team to do things their way sounds like a Half-Arsed Agile – gnat Jun 26 '14 at 14:52
  • 2
  • 8
    @Mark Richman: Waterfall? You do not have development cycles of 1-2 weeks in waterfall. I think he's no idea what he's talking about. – Giorgio Jun 26 '14 at 16:19
  • 2
    @gnat: if the team is not particularly high-functioning (and it sounds like this team fits that description), you could view this as the PM guiding the team to develop a more effective development strategy. Perhaps the PM feels that constantly delivering untested code is not good for the business. Agile doesn't necessarily mean "let the teams do whatever they want", there has to be some boundaries until a team is mature enough to decide for themselves. – Bryan Oakley Jun 26 '14 at 16:43
13

If you don't test a User Story (US) and verify that the acceptance criteria are met this story is not done. If its not done this US goes to the next sprint of course. And if all your US are in this state you sprint has ended with no value added to the project. From the client point of view I cannot distinguish this from the entire development team going on vacation.

One of the lean principles (agile doesn't end with scrum) says "quality is built in". Something is only done if it meets the quality criteria you define. This is crucial to have a real agile process, ending spring with zero value or separate testing from development are symptoms of a big problem.

There are a lot of things you can do:

  • automation is key to success. At least at unit test level, and some other practices like CI are important too. This is not enough, but if done well these types of testing result in few or no bugs discovered in manual testing (usually minor UI things). If you have dedicated QA people they can be the ones who automate the acceptance testing, and some of this automation can start before you finish a sprint.

  • Look at the size of your User Stories. If you have a US that is finished the two first days of the sprint the third day a QA person can test it. In my opinion having small (SMART) user histories one of the most important things to success in agile development, and a lot of people seems to no t realize this.

  • Collaboration between tester and developers is another key part of success. In my previous project when a US its finished by a developer a QA person do "pair testing" with the developer (can be manual, can be via launching some automated, or better, both), this works pretty well.

8

The essential problem is that you have programmers who code but don't test and testers who test but not code.

Solve that problem and you will not have this problem, and an arguably more efficient team.

One way that worked for me in the past was to suggest coders and testers to pair on stories with the explicit task of delivering a fully tested story. Together with that I've erased all forms of "dev complete" thinking: no "dev complete" columns on the scrum/kanban/trello board, no "dev done" attitude by coders.

What happened was:

  • Pairs were responsible for delivering stories and they would both fail if a story was not completed. They were treated as responsible professionals in charge of delivering software and they did, in most cases.

  • There was much less testing work done because testers were exposed to unit and integration tests, so they didn't repeat the same test manually.

  • Some testing got automated when the devs understood better what the testers needed.

  • Some people got upset.

  • Stories got delivered quicker on average because the code-commit-pull-test cycle became almost instant

Of course, this is only my anecdotal experience, but you might want to try that yourself if you can.

In your case, given your comment that testers and developers are authoritatively separated in your company, the message seems clear to me. There's an obvious barrier to communication and collaboration put up by company rules.

This is a communication problem, not an agile problem. Adopting an agile methodology is simply making it evident. Silo'd teams are a known management anti-pattern, so embrace the non-adaptability of agile in this case!

  • 2
    This organization has created clear boundaries and roles for "developers" and "testers", and n'er the twain shall meet ;) – Mark Richman Jun 26 '14 at 18:45
  • So, change the rule! – Sklivvz Jun 26 '14 at 19:04
  • 3
    @MarkRichman in one of my past jobs there were also clear boundaries in roles of "developers" and "testers", but that organization didn't put n'er shall meet block for them to communicate (what a lame idea btw!); I recall doing sprints in pair with "assigned tester" and it went great. Does your company simply separate roles or additionally sets a communication / collaboration barrier between engineers accomplishing these roles? – gnat Jun 26 '14 at 20:36
  • 1
    "The essential problem is that you have programmers who code but don't test and testers who test but not code.": Huh? Why is this a problem? A programmer should, well, program and a tester should test. Furthermore, you need some minimal feature that is implemented before you can test it: you cannot parallelize two tasks if the output of one task is the input of the other task. – Giorgio Apr 11 '15 at 10:08
  • @Giorgio that's a waterfall view. In agile, people who can deliver value independently are greatly favoured. There is no reason why development and testing should be separate professions. It's a choice, respectable, but poorly suited to agile development. – Sklivvz Apr 11 '15 at 10:11
4

The actual role of your QA is close to acceptance testing. I would imagine this to be done by a separate team, which acts more as product owner rather than a part of the development team.

Example: during a sprint, you need to add a feature which enables to filter search results by different criteria. You already have your search mechanism implemented, but the results are ordered alphabetically.

  • During the sprint:

    1. The team drafts the design of the new feature and the impact on the actual code base.

    2. Developers write unit tests which ensure that the ordering is working as expected, and at the same time writes the actual code.

    3. The new feature is thoughtfully tested to ensure that it doesn't break anything (regression testing) and that it is working as expected (unit tests).

    4. If possible and appropriate, which is not the case in most projects, a product owner (and so your QA team) can constantly evaluate the new feature in order to prevent the team going in the wrong direction. This requires continuous integration with dozens of builds every day.

  • After the sprint, the product owner evaluates the new feature to check that it corresponds to the needs of the customer. Your QA team is actually here, after the sprint ended.

I believe your actual issues are the following:

  • Scope. A sprint concerns your team, and your team only, not your actual QA which acts more as a product owner.

  • Testing. The fact that you have a QA team doesn't mean that all you need to do is to write code. The job of your team is to deliver a feature which works as expected, not to throw out code for the others to test. If you rely on QA team to do the testing for you, this will increase the overall cost, since bugs will be fixed one to two weeks later instead of being fixed nearly instantly.

  • I actually think that much of this organization's (to which I am new) issue is that there is little time spent up front analyzing requirements and defining a solution that can be decomposed into small atomic units. With the project and team's current state, I think the current sprint should have been nothing more than an analysis sprint, where the deliverables are PBIs complete with tasks and test cases. However, they seem to focus on the money they pay me every hour, and for every hour I'm not "hands on keyboard coding" they are losing value. – Mark Richman Jun 26 '14 at 18:48
  • @MarkRichman for every hour they pay you to type nonsense into the codebase they're losing not just the hour they pay you for, but all the hours it takes to get the nonsense out of the codebase. – Móż Jun 27 '14 at 5:48
4

if all or most of the coding is not done until the end of the sprint?

Why is it not finishing sooner? This key limitation is the source of your troubles, and I've seen two approaches be successful. One fits well into the agile approach (but not other common practices) and the other taints agile a bit (but is more common).

The first is that you don't code until the end of the sprint. Actually writing code is a relatively small part of development. If you finish about half way through the sprint, that provides plenty of time for QA to do their job. It also leaves you plenty of time to write documentation, clean up technical debt, do design for backlog items... All of the other stuff that you need to do for a quality product. The one downside to this I've seen is that it is nearly impossible to get the functionality and the unit tests done that quickly. Personally, I think it's completely fine to do unit tests after letting QA start to take a look at functionality, but TDD advocates (and others) will disagree.

The second option is to have QA operate a sprint behind the development staff like your PM hates. I also tend to dislike this. It eliminates the concept of "releasable product" at the end of the sprint, even if you have an escalation process for your releases. Worse, developers will be focused on "new" things when QA comes to them with questions or bugs from testing. Devs are also more unlikely to fix bugs in this arrangement. But I've seen it produce quality software on time.

  • 1
    I'm used to having QA be a sprint behind in their testing. The folks here want to see the entire SDLC complete every two weeks. I just don't see how that's possible. – Mark Richman Jun 26 '14 at 17:29
  • 5
    @MarkRichman - Why not? 1 day for planning, 5 days for coding and 4 for unit tests, documentation, bug fixes, and design for next sprint. The challenge is not really getting it done, but to be disciplined enough as a company to do a small amount of work well. – Telastyn Jun 26 '14 at 17:35
  • 1
    because they will focus not on the 5 days I'm coding, but the other 5 days I'm not. I would certainly fill the other 5 days with coding, but since they desire to have all coding tasks "soup-to-nuts" complete (including testing), it's just not consistent with the arrow of time's physics :) – Mark Richman Jun 26 '14 at 18:51
  • 1
    @markrichman - good, then it should be easy to point to all of the other things that aren't coding that you need to do to actually be done. – Telastyn Jun 26 '14 at 19:13
  • well, I also discover additional work that needs to be done in order to complete the work committed to during the current sprint. This forces other work to go unimplemented for that sprint. This is fine, and I think is in the spirit of agile, but I was told to never add anything to the current sprint, and add these newly discovered (and completed) tasks to the Product Backlog, which doesn't feel right to me. – Mark Richman Jun 26 '14 at 19:24
1

The Scrum Guide requires that teams be cross-functional. All team members are considered developers, irrespective of their individual specialisation. Silo'd individuals (coder, tester, qa, ux, etc) are unhelpful in Scrum. Team members help each other wherever they can. There is no concept of 'passing the item to qa'.

In your situation, it sounds as if you may have an estimating problem. When estimating product backlog items, you need to consider all activities, ie: Coding, Testing, Peer Review, Deployment, Integration - whatever your definition of done demands.

As a rough rule of thumb, expect to bring between 5 and 15 product backlog items in to a sprint. This gives you an idea of how large each product backlog item should be. It also gives you an excellent chance to get work 'done' within the sprint.

Finally, the task of the team is to move one product backlog item to 'done' and then move on to the next one. Sometimes, doing this means that people are treading on each others toes and so it makes sense to spin up more than one product backlog at a time. Your guideline though should be to reduce your work in progress (WIP) and move product backlog items to done.

0

Testing and coding go hand in hand. You could schedule it module by module. Once the module is finished, you could supply it to testers. This whole scenario also depends on which phase of testing you are working on. Spiral SDLC model looks good. In that, simultaneous testing and coding is convenient. Another approach could be V model.

  • I agree with you, but the "powers that be" seem to be purists about anything other than completing the whole SDLC in a single two-week sprint. Anything other than this methodology seems to be considered waterfall. – Mark Richman Jun 26 '14 at 15:07
0

My answer, which is probably quite strange at first, would be: you do not find time to test because you think testing must be done on the side effects of code. And with side effect I mean the computer science term:

a function (...) is said to have a side effect if, in addition to returning a value, it also (...) has an observable interaction with (...) the outside world.

I brought up the quote to emphasize the "interaction with the outer world" part: you want testing to happen on the UI as it is printed on the screen ("[to start testing] isn't really possible since you generally need a functional UI to record or create automated tests from").

Other answers have told you to automate this "acceptance testing", so that it can happen quickly. This is right, but does not fully address your original problem: what if there is not enough time to write those acceptance tests?

You have to let go your view of testing once the code has interacted with the outside world, i.e. it has printed something out and expects some input in. The problem with side effects is that they are, in effect, untestable. This dawned on me while reading an interview with Guido van Rossum, where he said that a statement that shuts down the computer can only be proved to work by executing it.

The solution to that "untestability" is to understand that, if you have proved once that the statement works, you can use it everywhere and rely on it to do its work. Isolate it in a function and test everything else.

Bringing that example closer to your question, the function is now probably a whole library or framework, and instead of shutting down the computer, it prints something out: a user interface. Keep the calls to it as dumb and as stable as possible, because once you enter that part of your application, you can only test via expensive acceptance tests, i.e. some kind of external observation.

Regarding the UI as "foreign territory" is actually a correct point of view, since you do not need to test the logic of a library that is not provided by you, and perhaps surprisingly, it is a realistic point of view: do you really expect to ever test that calling e.g. MyWidget.draw() does what you expect, to the level of a single pixel?

This is not to say that acceptance testing is not important, or that it can be skipped. It is there to stay and automating it, as other answers suggest, has enormous benefits. But if you want to find time to test and code in the same sprint, try to keep your code as side-effect free as possible.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.