How does agile estimate the time required for a "research phase"?

Question

According to canonical agile development, what is the appropriate amount of learning that an engineer should engage in before implementing a solution to a problem?

If an engineer knows she is too ignorant of a given topic (say, compression) to start implementing, she goes and learns what she thinks she needs to know, and then implements a solution.

Clearly a deeper understanding of said topic could result in a better implementation, but at the cost of agility. On the flip side if an engineer learns too little about a topic, there is a danger she will implement a significantly insufficient / broken implementation that negatively affects customers in the immediate or near future. In some cases, not a lot of research is required, and in some cases developers are faced with a topic even the senior-most developer knows little to nothing about, so the answer to this question includes an element of scalability to meet both situations.

Towards which end of this scale, if any, do guiding principles of agile lead an engineer? Or does agile leave this as a sliding scale, where the answer may be different in any given situation?

In other words, I'm looking to understand not when/how inside agile development research happens, but moreover how to estimate a "good" amount of time to spend learning a topic in the research phase before moving on to implementing a solution.

Answer 1

An agile team may consider a research spike when learning is required.

But that team should respect a lead developer's judgement when considering the length of time that the research should take. Indeed, the research spike's duration is entirely dependent upon the complexity of the targeted user story and the skill level of the team.

Answer 2

A basic theme running through Agile is the concept that the team should do just enough of 'x' to feel comfortable and responsible moving forward, but not more. This applies to many aspects of the software development life cycle such as design, documentation, testing, and research.

Putting an arbitrary and rigid time box around any of these ("spend up to four hours coding this functionality, but no more") is not very Agile. What if, after four hours, you don't feel comfortable that the functionality is complete? It would be irresponsible to move on to the next feature.

Likewise, consider research time ("spend up to four hours researching this functionality, but no more"). What if, after four hours, you don't feel comfortable making a design decision? It would be irresponsible to commit to a design the team doesn't feel comfortable with.

Clearly, however, one doesn't need to master all features of a given technology to be comfortable enough with it. The researcher should spend just enough time to feel like a good, responsible decision can be made; no more, no less.

Answer 3

When I worked in Agile, we used something called "Technical Spikes" to handle this type of work.

The concept is simple: You put a period of time in your sprint to research the technical issue at hand (such as what it will take to work with a certain technology that is new to your team). You then assign this to someone with an estimate. This estimate becomes part of your sprint plan.

This works really really well. Or at least it did in the context of how we were using it, which was mostly to research using a certain API to do a task. Certainly there are limitations - it would not work well if the learning task was too large (like learning c++ from scratch or something crazy like that).

Answer 4

"Agile" doesn't say anything about this specifically, but it does specifically say that things like this are best decided by the team. If it did dictate things like this it wouldn't be called "Agile".

The Agile that can be specified isn't Agile.

Answer 5

The speed of Jr. developers vs Sr. Developers is taken into account for the overall velocity of the team. So if a single member is lacking in knowledge, this is no different and should reflect itself very quickly as the velocity adjusts. With senior level developers on the team, and knowing who is most likely to work on a particular task, then the estimation should account for the ramp up time, and not have any noticeable effect on velocity at all.

If the whole team needs to learn the topic, then senior level developers should consistently predict the amount of ramp up time based on the complexity of the topic. This is due to prior experience of having to learn things on the job. In other words, its a learned ability.

In the end, worst case scenario is that your velocity will fall off unexpectedly and your team will need to readjust and lower the expectations from sprint to sprint (if you have those) so that expected velocity is agreed upon by all parties. Once this learning curve has been overcome, then velocity will increase and adjustments in commitments can move upwards again.

Answer 6

Just because you're agile doesn't change the implementation process, it just shrinks it into 2 week chunks. Include research/prototyping into your estimates, and note the added risk due to unfamiliarity.

You might tend towards less research since ideally refactoring is more common/easier, but it depends on your environment.

edit: After doing some research based on the first comment, it appears as though it varies. Some teams do not make stories available until they're designed, which likely involves research/prototyping. Slightly related question.

Personally, I've always seen/used the view that tasks are 'the steps needed to complete the story' and thus research should be spelled out there (or at least included in estimates if expected to be short).