1

We have a story "Scale service to X 000 transactions per second". It's unclear how to do that and how much time will it take. Usually in this case you can define a Spike to find out what obstacles you will face during implementation, what steps have to be done etc., so on the next Sprint planning you have enough information to define tasks properly and estimate the story.

However in case of our story it's iterative process. You run load tests, see that goal is not reached, profile, optimize something, run load tests again, profile, optimize, ... - research-implement, research-implement many times.

How to handle this type of stories?

migrated from workplace.stackexchange.com Nov 30 '17 at 18:04

This question came from our site for members of the workforce navigating the professional setting.

  • 1
    That's not a story, that's an epic. – Bryan Oakley Nov 30 '17 at 23:17
  • The way I learned it was a spike is research, a tracer is research + production code (when possible). So a tracer may not always result in a solution, but it 100% covers the spike and any attempts (obviously gotta guess very relatively on efforting). See bigagiletoolkit.com/B2B/Tracer_and_Spike.aspx – RandomUs1r Nov 30 '17 at 23:46
4

How to handle this type of stories?

A spike is the best thing you can do to address a story that is still needing to be refined or is more iterative than a traditional story.

There is nothing wrong with using a spike for this type of work in the short term (iterative style stories such as optimizations or research tasks for example), that is exactly what they are for.

The other thing you may wish to consider is to put this style of work on a Kan Ban board if it appears to be ongoing.

I found this article insightful on the topic of stories: Writing good Stories

  • If we put spike to one sprint and implementation to another, we will need 2 sprints (1 month if they take 2 weeks) for each iteration. If we need 10 iterations to reach the goal it will take 10 months that is not acceptable. From another hand, if we put spikes and implementation to the same sprint we have no clue how long will implementation take. That's the problem. – Marboni Nov 30 '17 at 14:30
  • 1
    @Marboni Unless you can break the stories down into tasks that you can then apply accurate estimates too you are spinning your wheels. Maybe the best bet is to move that work to a Kan Ban board. – Mister Positive Nov 30 '17 at 14:31
  • 1
    @Marboni Your spikes shouldn't find just one thing to work on. They should be trying to find as much as possible, and, to the extent feasible, should prototype the changes to get an idea of how effective they are. The goal of the spike is to discover changes that meet the performance target, not to discover just some improvement. The spike should produce one or more stories that you believe will lead to meeting the performance target, otherwise it has failed. You should not need a spike every other sprint. – Derek Elkins Dec 1 '17 at 1:58
1

You can break that goal down into multiple tasks. Assuming you've not done load testing of this service previously. You need to build a load testing harness to generate load. Trying to understand what realistic load looks like. You need instrumentation of the application to understand where the bottleneck is. That gets you a good chunk of the way into the load testing segment and then you'll have identified a couple potential sources of where the bottleneck is and can iterate from there. Those first three tasks can be estimated independently and done independently.

From there the iterative portion of making changes seeing their effect on the system can be better judged since you will have information like the load test itself takes so long and making a build takes so long so you can only try say 20 ideas a day. It will also give you information like I need to double the current load vs 10x the current load which will inform you about the scope of necessary changes.

  • The problem is that Xk TPS and latency are hard requirements. We can't start from 10 TPS and scale - we will have no customers accepting this small throughput. We are not a startup hoping to capture millions users - we are a backend service knowing how much load it will get from other services. – Marboni Nov 30 '17 at 15:01
  • Right but today you have an implementation of the service that does some number less than that, understanding the delta of change can help you understand how much work there is to do. If this is early in the process of building the new service and you don't have an implementation yet then you build the test harness with a skeleton app and setup the related automation to say if someone wants to make a change that would prevent this requirement then disallow it. – Sign Nov 30 '17 at 15:11
1

Like MisterPositive says, not all work fits the Scrum methodology.

Don't spike this in your regular sprint cycle - put some resources parallel to your other ongoing development into identifying the need first (if you're happily managing 10 TPS you might not need to worry about 10 kTPS right now).

Then those resources need to identify bottlenecks and research solutions, and the work to fix those bottlenecks can then be estimated into a regular sprint.

Absolutely does not spike to sprint to spike - it will wreck any velocity your regular sprints have - only put things into sprints when you are confident(ish) in the estimate (e.g., we don't include most things that are 13+ and end up breaking them down)

1

This is not a lot different from any other project. You have a large task that will be accomplished by doing a bunch of small tasks. Whether the small task is to create a database, develop a UI, or perform load testing is somewhat irrelevant for the purposes of planning.

In order to do this "story" (which is really an epic, but could also be looked at as a feature or product or project), you need to break it down into smaller pieces just like you would with any other project. Then, you prioritize the pieces, then start working on them.

For example, the first few stories might be to do the following:

  1. develop load tests
  2. run load tests to generate a list of candidates for optimization
  3. prioritize the candidates

At that point you might have a dozen or so candidates. The next story would be to work on the highest priority candidate, then the next, then the next.

  1. create first optimization
  2. create second optimization
  3. ... and so on

That's not the only way. You could, for example, set up a story for "create and run load tests", then "optimize biggest bottleneck", then just repeat those stories over and over.


That being said, not all types of software development or teams are a good fit for scrum. It's impossible to say what's best for your organization. If you have a top-notch team that can simply go into a room, lock the door, and be highly productive for two weeks, scrum is overkill. If you have a team full of less skilled, disorganized people, you may want to stick as close as possible to scrum.

The bottom line is to get the work done. If scrum helps, use scrum. If it doesn't help, find what does. The most important parts of scrum are the transparency and continuous improvement. The stories usually help with the former, and the retrospectives usually help with the latter. In between there's a lot of room to do whatever works best for your team.

1

I recently had a similar story to this. What you should understand is that with your current knowledge, you see the problem as requiring a series of random changes until you get the desired results, but you don't have to be in this state forever.

What I did was create a first story which had as its output a detailed list of concrete things to try. Then I had enough background knowledge to estimate how long it would take to try everything on the list, as well as which ones were most likely to work.

During the course of the trying stories, I discovered a couple more things to try, but for the most part I knew the upper bound of the effort at the end of the spike. There's still some uncertainty because the first thing you try might work, but the upper bound is much more accurate at that point.

You really will be a more knowledgeable person on a topic after a spike, if you do it right, and you should trust in that.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy