How much detail about a user story can a developer expect?

Question

The biggest drawback of agile development I have experienced is that people not involved in development focus on the mantra that a user story (3-10 ideal person days) should not contain more than 1-3 sentences like:

As a customer, I can use free-text search so that I can find the products I'm looking for.

Giving this sentence, project managers expect me as a developer to commit to an estimate and develop the story. They assume that agile development means that sentences like this are all they have to provide to developers.

I won't blame them because the well-known literature about agile development creates the impression that this would actually work. I've read something like 2 pages in natural language per story in "Planning XP", but that's it. Because "working software" is favored over "comprehensive documentation", this topic seems to be generally avoided.

The reality is, of course, that if the developer is given the chance to do so, an interview with the customer brings up a long list of requirements that the customer has about the story:

• We need boolean operators like AND and OR.
• We need fuzzy search an all terms.
• We need to search by single words as well as by phrase.
• We don't want to find products that meet criteria X, Y, and Z.
• We want to sort the result. Oh, and by the way, the user can select the sort criteria in a combo box with options a, b, and c.

So you see that I'm not talking about technical details or software design or even implementation details. It's pure requirements. The longer we talk the more the customer realizes that there's actually quite a lot to say about what they want.

But often enough I find myself in the situation that such information is not provided or in very shoddy fashion. Neither is it possible that I do the interview, nor does the person that would be in the position to do the interview provide me with a result of it.

Sometimes, managers even come up with technical details like "we want Lucene search" but they don't want to think about if they want find only product names or also product descriptions. Sometimes I think they are just lazy ;)

For me, this is the top issue in projects I work in (e-business web application, 500-2000 person days per project). I've addressed this problem often enough, and managers are aware that most developers have a problem with the situation. But they believe that developers are just too much "perfectionists". They seem annoyed that developers "always want to have everything specified".

Due to the lack of generally acknowledged numbers it's hard to argue. Everybody knows how long an iteration should be. But nobody can tell how much requirements are needed to estimate and develop a story.

Do you have some reference?

Answer 1

You're missing the point of agile a little bit. What you are calling a user story, I see as at least six: one bare-bones search, and one for each of your bullet points. By all means, make enough plans to avoid painting yourself into a corner that's going to be expensive to get out of, but the whole idea is that you provide the minimum needed to deliver some value, and do it quickly enough to get rapid feedback.

When you divide a story up like that, it not only makes it easier to estimate, it allows the product owner to prioritize in a more fine-grained manner. Certainly they like the ability to sort the search results, but maybe it's not as important as another item on the backlog that's completely unrelated to search.

Also, on the idea of programmers needing everything specified, try to look at it from the customer's point of view. A lot of times, it's like you're going in to buy a car, and the salesman is asking what color you want for the windshield wiper knob. A lot of details we might find important are completely irrelevant from a customer point of view. I've worked where requirements are highly over-specified, and trust me, it is not very fun. The kind of latitude you're complaining about, a lot of programmers would love to have.

Answer 2

Sounds like the first issue is that you aren't supposed to apply time estimates to user stories. You're supposed to take a story and apply "story points", which are a general estimate of complexity from 1 to INFINITY. Story points are often run something like 1,2,3,5,8,13,20... (Every organization has its own rules.) They generally apply something like:

1 - You can do it in your sleep and it is hardly worth implementing. 2 - You understand this, and can get it done quickly with little risk of overrun. 3 - You understand this, but there might be a surprise or two. 5 - This is going to a little research, and has some small amount of risk. 8 - This is a large task, needs lots of research, and may not fit in a sprint. 13 - This is huge, and definitely won't fit in a sprint. There's massive risk. etc.

Generally, any user story that is an 8 or above needs to be broken down into smaller stories.

If you don't have the information to do this, you definitely should throw it back at the project manager and say that you need more information.

You really should only be estimating time once you have accepted the story into the sprint, but even then, there is less emphasis on this. The idea is that once your team gets used to the pointing process, you can measure its rough output per sprint in story points, and plan that way. You don't want to be planning on a shorter timescale than the sprint. The idea here is that if you're breaking down tasks correctly so that the multiple stories fit in a sprint, and are in the 1-5 story point range, it means that they are well defined enough.

Also, it sounds like the PMs at your company don't understand what a "story" is. A critical part of a "user story" is the exit criteria. The exit criteria is a short sentence or two that describes unambiguously how it can be shown that this storage is completed. Ideally, this should be something that your QA guys have said "yes, we can test for that". The important bit is that the PMs have to understand that a user story is complete when the software meets the "exit criteria". "But we didn't want that" doesn't cut it. If they didn't want what was delivered, but it matched the exit criteria, they have to enter a new story.

There is certainly an element of "training the PMs" here. They have to learn that vague stories result in large story points, and that if they defined the story to ambiguously to get what they want, they have to do it again.

Obviously if the stakeholders aren't gathering enough information, then you have to, and if you have to, then that's a lot more work, isn't it? Long before my agile days, I had success by giving very large estimates, and explicitly saying that the estimates were so large to allow for the risk caused by the lack of information. I had to assume the worst case for all questions, and estimated based on this worst case. I found that managers were more willing to give more details when they saw it resulting in estimates going down.

This isn't gaming the system...this is perfectly valid. If you don't know whether it is "A" or "B", you estimate based on which gives the largest estimate to cover your ass.

Answer 3

In my experiences, many of the changes or projects I'm working on deal with this very thing. Custom X wants something and they have an idea of what they want, but they only give you a small email worth of requirements. That's mostly because the client doesn't 'exactly' know what they want. That's why most of our client service department's job is fleshing out those customer demands and filtering the needed information while also predicting what the client REALLY is going to want, or what they really need.

Say a client (for me) wants a section of our web application to return them a list of all phone numbers. They never specify if they mean physical ones, logical ones, ones that are assigned to a person or a location, ect. They simply want all the phone numbers. As a developer, I can sit there and think of a dozen or more questions that I would need to ask the client, much like you have. But, as you say, that's not possible. That's why having a good client services department that knows the product and the client is invaluable.

When that kind of call comes in to our client reps, they are able to elaborate on it with the client, knowing what they need to ask them to get the right questions answered. They also have the forethought to know what the client has asked for in the past and they know enough about the systems that we develop that they can say yes or no to something without even asking the client.

Sure, you'll have a lot of cases where the client will still need something else that both you and client services missed, but that's always going to happen. Your goal, and the goal of client services, should be minimizing the lag time between you developing something and you getting it back from the client with changes that need to be made. And this just comes down to communication and training with your client services.

Maybe it's not as feasible for you as it is where I am, but having a good line of communication and trust with your client reps will almost always help you by degrees, and it will reduce your frustration and increase the client's satisfaction. Also, you can more easily give a time frame for your projects rather then shrugging your shoulders and saying "I don't know the full scope of the project, so I don't know how long it'll take". We're having the same problem here, and better communication and training is what's helping us create reasonable deadlines and hit them consistently.

Answer 4

Customer

I want to search products

Product manager I have analysed customer's story and came up with the following requirements. Each requirement has been recorded as a seperate user story.

• Search for products
• Sort result
• Filter search results

Developer I have received user stories from a product manager. I have analysed each user story and came up with a list of task for each user story.

• Search for products
  1. Task 1: Database changes
  2. Task 2: Server side changes
  3. Task 3: Front end changes

Customer, product manager and developer are all stake holders in this process. They all need to contribute to the analysis process before the work can commence. Please note, that this is very simplified example.

Our user stories are analysed and refined in the following order (with some variations of course):

Help Desk --> Product Owner --> Product Manager --> Department Leads (senior developers, qa leads, etc) --> Developers

Once all the relevant stakeholder have contributed to the analysis process, we can estimate how long it will take us to deliver the story. Estimation process that we follow is based on velocity and expertise of individual developers.

I'm not saying that this is a correct way of doing things, but it works really well within our organisation.