12

Most of the literature on agile seems to be biased towards CRUD type business applications where the user is pretty much aware of what is going on behind the scenes. (That's fine because most of the code being written probably belongs to this class.)

For this type of application the relationship between user stories (requirements) and development tasks is mostly straightforward: Just split up the user story into a few tasks.

But there is another type of application where most of the code has to deal with complex processing that is not directly visible to the user. Examples would be:

  • Compilers
  • Image analysis systems of self-driving cars
  • Fluid flow simulation systems

Here it can get really difficult to relate tasks and user stories. Are there techniques to overcome this issue or is it just something we have to accept and make the best of it?

  • 6
    Not the downvoter, but I'll assume it's because the question is based on a false premise. IMO complex systems are even more suitable for an Agile style development by virtue of the fact that they are more complex. The more complex the system, the more likely it is to have emerging requirements. Agile SwDev was created to better handle emerging requirements. – RubberDuck Jul 23 '17 at 12:08
  • 4
    @RubberDuck: "I'll assume it's because the question is based on a false premise.": IMO, this would motivate an answer in which the false premise is explained, not a downvote. – Giorgio Jul 23 '17 at 14:10
  • Whether the use understands the logic or not is totally irrelevant to the agile process. It is up to the a team to map a user story to an implementation AND to estimate how long that will take. If it is complicated and/or a lot of work, the team can split up the story into smaller ones. The type of logic does not matter. – Martin Maat Jul 23 '17 at 23:30
  • 2
    "Most of the literature on agile seems to be biased towards CRUD type business applications" – And most of the literature on Java seems to be biased towards printing the string "Hello World" on the console (or alternative computing the nth Fibonacci number). That doesn't mean that's the only thing Java is good for. The reason is the same in both cases: it's tough to explain realistic examples in a blog post or tutorial. [Note: this is a hyperbolic comment trying to illustrate the basis for the false premise.] – Jörg W Mittag Jul 24 '17 at 20:10
  • 1
    Agile doesn't require tasks and user stories. You can use any form of specification you wish. The whole point is flexibility. – Frank Hileman Jul 24 '17 at 22:09
9

This turned out to be longer than I had planned (it had started as a comment), but I hope the added length/details are helpful and you find them justified.

Agile is not specific to CRUD apps

Most of the literature on agile seems to be biased towards CRUD type business applications where the user is pretty much aware of what is going on behind the scenes. (That's fine because most of the code being written probably belongs to this class.)

I think this is because it's easier to create easy-to-follow examples of this type, not really because the methodology is aimed at those kinds of systems. If you create a not-so-easy-to-follow example, you risk getting the reader stuck trying to understand the example when your point was supposed to be teaching the reader about agile concepts.

User Stories != Requirements

For this type of application the relationship between user stories (requirements) and development tasks is mostly straightforward: Just split up the user story into a few tasks.

A user story is not the same as a requirement. True, there can be some overlap depending on how 'high-level' the requirement is, but generally not the same. I get the impression that you're running into the same pitfall a lot of my former managers fell into: thinking about user stories simply as synonyms for "requirements", which is similar to when SVN users try to transition over to Git, but keep thinking in terms of SVN. (They then run into problems due to the bad starting assumptions.)

IMHO, a key difference between requirements and user stories is that requirements specify, in detail, how certain system components ought to behave; they're specifications that include inputs, outputs, assumptions/pre-conditions, possible exceptions raised, etc. They focus on what the system does.

OTOH, user stories focus on the expected outcome for the end-user without trying to create a detailed behavioral specification for system components. They focus on the expected user experience.

What I used to do, and this was a practice my team adopted, was to break down user stories into tasks. Your tasks could be as specific or vague as you wanted/needed them to be, but they were meant to be your progress indicators for actual work done towards getting the story to a done-done state.

Example

I roughly recall a US I worked on years ago where users needed to self-assign test cases so that everyone in the team was aware of which TCs they were working on to avoid duplicated efforts; the UI was a(n internal) web application. The user only saw a button, but the story was divided into several tasks that included some technical implementation details, etc.

User Visibility

But there is another type of application where most of the code has to deal with complex processing that is not directly visible to the user.

Is it possible to make it visible to the user in some way?

Consider a GPS. When you've missed your turn, you won't see the actual route re-calculation process, but the user does receive some useful feedback (e.g. "Recalculating...").

Compilers may display warnings or errors, or include new settings/options in the GUI for users to see that something new has been added. I'd think the users for compilers would be programmers, right? Wouldn't they see support for a new standard added?

While supporting a new standard would likely be at the feature level and would need to be broken down into user stories, have you made sure that, at least in some cases, you're not trying to use stories when you should be using features instead?

Image analysis in a car could be phrased in a way that lets the user know that the chances of ending up in a car crash have been reduced. For example:

As a passenger in a self-driving car, I need the probability of the vehicle causing an accident by crashing into an unrecognized object to be as close to zero as possible, so that I can travel more safely.

That US captures, at a high-level, things you'd normally have to specify using a combination of functional and non-functional requirements -including security, safety, etc.

However, a requirement might be more about the system; e.g.:

Function abc in component A must have the tolerance threshold value decreased in the image comparison algorithm to better detect objects moving slowly.

To me, this would easily be a task under the user story I mentioned above, titled something like: Decrease tolerance in function A.abc and then include other relevant details in it.

For a fluid simulation system, you could even have a progress bar that provides feedback about background tasks the system is performing, if this makes sense. (There's always a way to inform the user of something, though you may want to avoid being spammy.)

I don't know enough about the particular domains you've mentioned to come up with better and/or more realistic examples, but if there's a take-away here is that you can use different ways to provide user feedback about something less visible that the system might be doing, i.e. there might be ways to make invisible things a bit more visible. (Even if it boils down to writing a set of release notes that documents how much faster the system's performance is now due to your efforts, etc.)

Relationship between Stories and Tasks

Here it can get really difficult to relate tasks and user stories.

Our approach was to keep user stories focused on what the request was, why it was made, and what things needed to be true to consider the US "done". The how was always left out of the US and left to the developer(s).

The developer(s) would break down the problem described in the US into a set of tasks that they would work on.

I'm saying this as someone who, for the most part, did back-end server-side programming, which is probably as "invisible" as you can get for the end-user.

Depending on what I needed to do, I'd sometimes use AJAX to show a simple "loading..." animation/gif so that the user knew they needed to wait a bit for something else to complete, without getting the wrong impression. Sometimes it was as simple as this. A task for this would be appropriate.

Different Paradigm, Practicing, and Experience

Are there techniques to overcome this issue or is it just something we have to accept and make the best of it?

Beyond accepting the paradigm shift, practicing, and accrued experience, probably not much more to say. I often saw people trying to take shortcuts through the process. I'd advice against that, especially if you're getting started. As you get more experience, you can allow some flexibility, but avoid undermining yourself.

Given your prior wording, you're still thinking about stories as if they were "renamed requirements", which I think is a false assumption. I think this is a symptom of a deeper issue regarding the fundamental differences between Agile and non-Agile approaches.

Secondly, I think you should accept that agile is a paradigm shift compared to waterfall, which means that, while the process has similar goals, they go about it in very different ways. (Think SVN vs Git, if that helps.)

Try to improve your current understanding of the conceptual differences between requirements and user stories and accept they're not the same thing.

Learning from your Sprints - Retrospectives

What I cannot stress enough is the retrospective between Scrum Master and Developers at the end of each sprint. That is the place where they discuss things that "went well" or "didn't go well" in an honest/transparent way, and what doable changes will be implemented for the next sprint to address the "didn't go well" points.

That allowed us to adapt and even learn from each other's experiences, and before we knew it, we had improved significantly as measured by the general consistency of our team's velocity.

  • "User Stories != Requirements": I didn't mean to say that the two are synonyms. Not every requirement is a user story. But I do think that all user stories are requirements. I view requirements as a hierarchical structure where user stories are one specific level of detail. Would you agree? – Frank Puffer Jul 24 '17 at 11:21
  • @FrankPuffer I think viewing user stories as if they're a different level in a hierarchy of requirements is basically mixing different concepts. On the Agile side, a hierarchy looks more like: Themes >> Epics >> Features >> User Stories >> Tasks. Requirements are usually divided into functional and non-functional requirements in the more traditional Waterfall approach, but I've not come across an actual hierarchy; that said, I wouldn't be surprised if someone were to recursively break down a requirement into smaller "sub"-requirements. In any case, you're mixing different concepts. – code_dredd Jul 25 '17 at 0:29
  • Requirements management systems like PTC Integrity treat requirements as a hierarchy. This can be an advantage when mapping requirements to a specification document. – Frank Puffer Jul 25 '17 at 6:59
  • @FrankPuffer Yeah, that's why I said I wouldn't be surprised. That said, I'm wondering my answer has clarified anything for you. Was the SVN vs Git analogy helpful? (It assumes you're familiar with both systems, which seemed reasonable in a software development context.) – code_dredd Jul 25 '17 at 8:46
  • Ok, I misread the "wouldn't" as "would". And yes, I find your answer very helpful and will probably accept it. I probably just need some time to get accustomed to the idea of not considering user stories as requirements. – Frank Puffer Jul 25 '17 at 9:55
4

Agile principles can certainly be applied in these cases. How?

  • Compilers can have new language features added later in separate user stories
  • Image analysis systems can have new features added as different image classifications
  • Fluid flow simulation systems can take different model aspects into account in their simulations

You don't have to eat the entire elephant in one bite. Agile just asks that you show you've cleaned your plate before the next serving of elephant.

  • Still I think that one or more user stories will remain that require lots of so-called base functionality. They will often not fit into a single sprint. How should this be handled? – Frank Puffer Jul 23 '17 at 14:32
  • 1
    You measure successfulness with runnable applications that customers can tests, see or touch. If that's the case, give to them a toy that generates the sense of progress in such way. For instance, you probably can't release a self-driving car in the first sprint, but you can make a demo of how your algo do recon people. Later, how It recon signals and animals. Later how It measures distances, sizes, etc... The self-driving car is far, far away in a remote sprint that who knows if it will ever happen. – Laiv Jul 23 '17 at 16:11
  • 2
    @Laiv: That would be nice but I am not sure if it works. In parctice, after the first few sprints, the software will not be able to do anything the customer can relate to. You will have technical progress but it will be hard, if not impossible, to communicate it to the customer. – Frank Puffer Jul 23 '17 at 16:29
  • 2
    Why? Because It was written on Stone by someone foreign to your project? I expecte Agile to adapt itself to my needs, not otherwise. If I say I can teach you to skate in 4 weeks and once on the 5th you still fail on standing, would that mean you are not learning to skate? Or just that It will take you a little bit longer? Agile manifesto was not written on stone nor the Scrum's rules are the Tend Commandments. – Laiv Jul 23 '17 at 19:07
  • 2
    The point of the sprints is to make your customer part of your iterations. To communicate with them using delivered code. Not plans and promises. It requires you to focus on breaking down the problem. It doesn't require that the first thing you deliver be set in stone. It requires that you prove you're worth paying every sprint. – candied_orange Jul 23 '17 at 20:42
1

I find that people who adhere to strictly to user stories will either just engage in a very silly exercise of coming up with far fetched ways in which back end technical changes impact the user (unbeknownst to the user of course because they're just some naive user and you're talking about complex changes in your data analysis pipe line or something of that sort) or they will just be at a complete loss for "how do we possibly organize this work!?!"

I think the obvious solution is to be more pragmatic. If the work is very technical in nature and doesn't have a particularly noticeable impact on the user, don't lose sleep trying to explain how it does. Just choose an obvious and simple way in which it can benefit users and then orient the story around the details necessary for the developers to do their jobs. I find it incredibly frustrating when a PO insists on not having technical information in story when it's absolutely necessary. It's just not a very wholistic view on what that artifact (the story) really is. Like they think it exists just for them, in most cases it's important to the engineers as well.

For most of these technical tasks there is some low hanging fruit with regard to user impact, whether that is improving efficiency so future deliveries will be speedier, improving performance, reliability ect. They're not really what people tend to think of when they think 'user stories' but if the business wants to understand why you would undertake technical debt or something to that effect, then these explanations are usually the simplest to provide.

tl;dr don't let a scrumnazi make your life extra difficult simply because they're too much of a square to adapt. Being adaptive is a core concept of agile after all. Strict adherence to Scrum or Agile usually flies in the face or pragmatism and practicality (what actually works best).

  • I'm all for being flexible, but frankly the user doesn't particularly care about the technical details, so long as their stories get satisfied. You don't have to be "strictly agile" to have good requirements; and by good requirements, I mean requirements that are each accompanied by an acceptance test that unambiguously proves the requirement is satisfied. The folks that are "engaging in very silly exercises" are obviously doing it wrong, but that doesn't mean they're following some notion of "strict scrum." – Robert Harvey Aug 16 '17 at 0:17
  • @RobertHarvey I agree the technical details are irrelevant to the user but my point is that the artifacts containing user stories have a broader purpose than just communicating how things work for the user (or they should at least). How does one enforce requirements related to architecture, extensibility, performance and so on if they write purely user stories? Taking a pure 'user story' approach incentivizes developers to write poor quality code. And it's not users reading the 'user stories' it's devs and qa, it's foolish to deliberately exclude relevant information because it's technical. – evanmcdonnal Aug 16 '17 at 1:03
0

I think the issue is in giving user stories a meaning they do not have. Scrum uses the term PBI, or Product Backlog Item, which I think is infinitely better. PBIs will often follow a user story format, for example, you might have a PBI like "Subscribers should be able to view their subscription details", but you also might just as easily have a PBI like "Create a stored procedure to get subscriber details".

User stories are a tool. They help you form feature descriptions and requirements based on putting yourself into the shoes of a user. But, just as a wrench is useless when you need to hang a picture, there's times when you may not need a user story.

That said, many teams actually just play fast and loose with the "user" part. They may have "user stories" like "As a developer, I need to be able to call a stored procedure to get subscriber details", essentially a "developer story" as it were. This is an equally valid option, but personally, I say as long as your can describe what needs to be done and come up with a set of acceptance criteria, it hardly matters if you have an actual user story behind it or not.

  • 1
    I don't agree with this except in very strange and rare cases. In Scrum the HOW is the purview of the development team, not the product owner. Yet the product owner should be responsible for PBIs. So a PBI like "create a stored procedure" takes away from the development team the right to choose the HOW. PBIs, whether user story or not, should explain what the business value is in doing the PBI. Creating a stored procedure isn't about business value, it's about implementation detail. – RibaldEddie Jul 27 '17 at 17:49
  • That's not the point. That was just an example. You could just change "stored procedure" to something generic like "a way". The point is that it doesn't have to necessarily be a user story. – Chris Pratt Jul 27 '17 at 18:42
  • the whole point of an example is to be an exemplar. If your example "isn't the point" then what is the point of the example?? – RibaldEddie Jul 27 '17 at 19:21
-3

These types of applications are exactly the ones where differing expertise is present and will develop further. The team members will have due to differing education, different hobby projects and different past work experience differing skills. Futhermore, if somebody develops a particular piece of code, the developer can be expected to be the one who knows the code best. So it might make sense to give further development tasks involving the same piece of code to the same developer.

In the most popular agile process, Scrum, there is planning poker where each task is assigned a difficulty level. The difficulty level is not dependent on the person who is doing that task according to the process. Then during the sprint, people are considered homogeneous so that each person is expected to be able to pick every single task and implement it. In simple CRUD like projects, this assumption holds. But in very complex and difficult projects, it certainly doesn't.

I wouldn't use an agile process for these kinds of projects. Your best choice is to avoid any formal processes and just use good project management. When deciding who implements a particular feature, consider who has the best skills needed for this feature and best knowledge of the existing code. No process is needed for this. You will probably want to write good design documents for all the features and keep them up to date. Note that I'm not promoting a waterfall-like model here: the design documents will not all be written at the start of the project; you will instead write new design documents as new features are needed.

  • 5
    Not really related to my question, but always letting the one with the best skills implement a feature can be dangerous. It hinders the spread of knowledge within the team. If maintenance is required and the expert has left the team or is temporarily unavailable, you will be in trouble. – Frank Puffer Jul 23 '17 at 14:37
  • @FrankPuffer For some of the kind of systems you listed, e.g. self-driving cars, if the expert leaves the team you're in trouble. Period. While it probably isn't the case that the coding should be centralized, it's also completely unreasonable to assume an adequate "spread of knowledge" to allow the expert's replacement in any reasonably short time-scale. These are people who will have spent over a decade researching the problem, and are presumably near the top of their field on it. You will probably need multiple people like this with different skillsets. Such projects are inherently risky. – Derek Elkins Jul 23 '17 at 19:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.