1

First of all, I found it a bit strange because technically, iterative means something that is done as a recurring series of steps - so how can one be purely incremental, when each increment will be implemented through the same series of steps?

Anyway, my main confusion stems from the word "iterative" - if the meaning is enahnced to mean "refining and revising work", where does it fit in the classical story development scenario: you develop story B, and once it passes the tests and meets your DoD and Acceptance criteria, it is Done. You added it on the top of the existing baseline. In the mean time, some other stories are finished, ideally Done and hence certainly not meant to be refined or reworked... Where is the iterativeness?

So, to me it does not seem different from the decades old incremental models (which some claim are examples of Agile models, some say they are not). Just developing features one by one, with early testing and frequent delivery of working increments.

Another interesting point: Agile is apparently said to be "one of the incremental models". However, incremental models are defined by quite a comprehensive upfront documentation...

11
  • 2
    However, incremental models are defined by quite a comprehensive upfront documentation... -- Can you clarify this statement, please, perhaps with an example or description of the documentation you're talking about? May 2, 2021 at 17:27
  • 2
    I find your question hard to understand. You ask about the meaning of certain terms, but you don't give us a lot of context where they are used. E.g. "Agile is said to be iterative-incremental". Who says that and what is the exact phrase?
    – Helena
    May 2, 2021 at 18:27
  • 1
    Oh come on, he's not asking a complicated point. He simply asks, how does the jargon of something being "done", and a picture of accretion, reconcile with an iterative principle in which a system is (potentially) never finalised and always subject to revision?
    – Steve
    May 2, 2021 at 18:44
  • 1
    Your question would be a better one if it were more about the SDLC and less about word definitions. May 2, 2021 at 19:30
  • 1
    @RobertHarvey Sorry, but what definitions used don't actually exist? I can agree with all of them. The incremental model is an SDLC variant of splitting requirements, it has nothing to do with agile per se(javatpoint.com/software-engineering-incremental-model)
    – John V
    May 3, 2021 at 6:08

6 Answers 6

6

It seems like you are mixing several things together, which is why you're confused.

Iterative and incremental development refers to how the system as a whole is built. Contrast typical methods that are associated with Agile Software Development and associated techniques and principles like incremental design, YAGNI, and refactoring with "waterfall" techniques that embrace getting all of the requirements upfront, then spending time architecting and designing for all the requirements, then spending time turning the design models into code, then integrating and testing the system. Instead of trying to do everything for the whole system up front, you'll be repeating the activities for each slice of the system.

When you look at individual units of work, just because you've developed, tested, and shipped something doesn't mean that you won't revisit it. There are a few different reasons why you may need to. One would be that you get feedback that what you built is no longer quite right and you need to revise that feature. Or perhaps you are building a related feature and need to revisit your design decisions to support both, yet keep a maintainable system. Maybe the team's technical maturity grows and an evolving Definition of Done means that there's now technical debt to be resolved.

10
  • Thanks, yes it could be because I look at the low level only. But then how is that different from incremental waterfall? Even in that case, with clearly defined scope and just delivering in pieces, I will be repating the steps for individual slices (though not really for requirements).
    – JCerm
    May 2, 2021 at 17:52
  • 3
    There's no such thing as "incremental waterfall." May 2, 2021 at 19:29
  • 3
    @JCerm I have never heard of "incremental waterfall". I'm aware of a few "modified waterfall" structures, like the sashimi model where the phases overlap slightly (such as starting design before requirements are finished), waterfall with subprojects (where you collect the system requirements, but allocate more detailed requirements as well as all downstream work to subprojects, until integration). Incremental seems to be the exact opposite of waterfall.
    – Thomas Owens
    May 2, 2021 at 20:08
  • @RobertHarvey Of course the Incremental Waterfall exists. Please do some research before stating such claims. From the book Beginning of Software Engineering (Stephens, 2015): "The incremental waterfall model (also called the l multi‐waterfall model) uses a series of separate l waterfall cascades. Each cascade ends with the delivery of a usable application called an increment."
    – John V
    May 3, 2021 at 6:14
  • @ThomasOwens Pls search for "incremental waterfall", there are many mentions also in the books available online. Why would it be the exact opposite - you still have a complete set of requirements, the only difference is that you repeat the cycle for a few increments, instead just once.
    – John V
    May 3, 2021 at 6:20
2

You're right that agile software development is not necessarily radically different from other software development approaches. The “radical” idea in agile is to Do Things That Work, which usually means adapting the process to make it work better, and to listen to the people doing the work (“people over processes”).

Under the agile umbrella, we have specific process methodologies such as Scrum. Scrum is definitely iterative and incremental, and suggests feedback loops of various levels. In a typical instantiation of a Scrum process, you'd see feedback loops like the following:

  • retrospectives
  • sprints
  • standups
  • TDD red/green/refactor cycles

In Scrum, the main feedback loop is the sprint. Each sprint has a defined duration and defined scope. At the beginning, we determine some tasks that should be Done during the sprint. At the end, we see what was Done. Did some things fail to get Done? During the retrospective, there's opportunity to understand why and adapt as necessary for the next sprint. Then we repeat. Since we might adapt between each Sprint – and in particular, the Product Owner may change priorities based on the experiences – it is not appropriate to plan in advance multiple sprints. One step at a time.

You are entirely correct that individual tasks (backlog items) in Scrum are not handled in an iterative manner. A task is either Done or it's not. This is considered important in order to manage clear expectations with the Product Owner, but also to demonstrate clear progress to the PO.

The entire point of Scrum is that many projects do not make clear progress because a lot of tasks are only mostly done. This delays value, and makes it difficult to close feedback loops. The idea is that with Scrum, there will be something Done each Sprint, thus delivering clear and predictable value to the PO. And since the PO can re-prioritize between Sprints, this maximizes the value delivered across the project duration.

When you say “incremental models are defined by a comprehensive upfront documentation” then this is wrong. There are processes without a focus on documentation that are clearly incremental, such as Scrum. It's also perfectly OK to prepare lots of upfront documentation in an agile project, if that documentation has value (though in many cases, a Spike is more useful to find a good design than to do lots of upfront analysis). This circles back to my original point: agile is about finding a process that works. In this viewpoint, being agile is about values that inform how you select and implement processes, and are not an inherent attribute of processes themselves. (Indeed, most Scrum instantiations are probably not agile in any meaningful way…)

Iterative/incremental processes were pioneered in environments with very high integrity requirements and thus a focus on documentation and testing. In particular, some incremental processes involve an entire Waterfall-like process per increment, including analysis and design phases. This isn't wrong, but this also isn't the defining feature of iterative/incremental. The defining feature is that functionality of the project is split across two or more increments, thus de-risking the project and allowing for early feedback to be integrated. This is broad enough to cover both defense projects that start with a 1000-page requirements document and a three-person web consultancy doing Scrumban.

19
  • Well, all definitions of incremental models state they are require detailed upfront analysis because all you do is breaking down the scope to a smaller chunks, which is not exactly what Scrum does, because the traditional incremental model is often referred to as the "incremental waterfall". Also I think you have not addressed how this iterativeness is then reflected in Scrum.
    – JCerm
    May 2, 2021 at 17:32
  • 2
    @JCerm: Can you provide a citation for this? This is actually the first time I've heard this assertion. Every Agile team I've worked in (including the current one) has started with a simple "wish list," and refined requirements as work progresses. None of them started with a detailed upfront analysis; you're describing waterfall, not agile. May 2, 2021 at 17:35
  • @RobertHarvey Could be found in varous Google books, but this one is the simplest one I could find: guru99.com/… The incremental model is typically used when requirements are well defined and well known. You then merely de-risk the project by breaking down the scope into increments, which is why I believe those who distinct between incremental model and Agile are right.
    – JCerm
    May 2, 2021 at 17:39
  • 1
    One of the more well-known incremental process frameworks is IBM's rational unified processes. Of course RUP allows for mini-waterfalls and lots of ceremony. After all, IBM is a consulting shop. They don't make money by delivering value, they make money by spending billable time.
    – amon
    May 2, 2021 at 17:45
  • 1
    @JCerm the guru99 article you linked to is not very well written; e.g. the advantages/disadvantages section directly contradicts itself on each line, as well as things previously stated in the text. May 3, 2021 at 5:38
1

I think your confusion stems from what you associate the word "incremental" with; I get a feeling that you're thinking about rote accretion of features (software simply growing by addition of new things, and piecewise delivery). That's not what's meant - it's about design, about deciding what to build and how.

The problem with waterfall is "big design upfront". This may work when the problem domain is very well-understood (e.g. there are years of research, how things work and interact is known, there are standard time-tested practices, etc.), but it's not iterative or incremental in the sense that pretty much everything is decided upon and specified in one phase. (Now, you could call it incremental in the sense that you're probably building it in multiple steps, in an additive process, but that's not in itself especially interesting.)

Agile recognizes that in many cases for business applications the problem domain is either sufficiently unique or isn't well-understood (in the above sense), and that doing "big design upfront" when you don't know enough about the subtleties of the domain leads to expensive projects that in the end don't do what clients wanted (you build "the wrong thing"), or do it poorly - assuming they weren't overly ambitious and were successfully built at all.

So agile opts for "some design upfront", and then some more design at each iteration, as the developers learn more and get feedback. Agile is trying to hone in on a good solution over time, starting small and retaining design plasticity for longer. Another aspect is that agile tries keep the product useful/usable at every iteration. It also strives to learn what the most important (core) aspects of the system are, and tries to push those to be implemented first (or as early as possible). This combination reduces the risk for clients because they can stop the project when they feel is appropriate (e.g, when they reach the point of diminishing returns, or when they run out of budget), and still get the most value out of it - they still end up with software that is actually useful for them. (Note that there's a business aspect to this as well, you have to have a business model that can support this approach; e.g. you can't have scope, cost and time all fixed, etc.)

The revising aspect is going to come from the the project team developing a growing understanding of the domain, and from frequent feedback - from uncovering new things, or identifying what was misunderstood from (or misspecified in) the requirements, or what was assumed and never explicitly said, etc. It's the moments that make you go: "Oh, so that's what you actually want!", or "Ooooh, this process actually works like this!", or "So, when you say 'shipment' you don't mean the same thing as when your colleague from a this other department says 'shipment'???".

So it's really about incremental delivery of value (clients find product usable/useful in some way after every iteration), and the ability to learn and revise on the go. (Or, if you want to say it differently, it's about being incremental and iterative simultaneously - which then implies that you have to be incremental in the particular way described).

In contrast, a waterfall project might spend a long time in a half-baked, work-in-progress state, and can proceed for a long time before any feedback is provided (or even possible).

1

Your story meets the acceptance criteria, so it’s done. You will never, ever go back to that story. Absolutely right. That doesn’t mean you never change the code, and you never change the application.

Say your acceptance criteria said there must be a green button on the screen. You do it, QA checks the button is green, story is closed. Next day a customer complains: The button should really be orange. Yes, they said “green” in a meeting, but now they want orange. You don’t re-open the closed story, you don’t re-visit the story. You create a new story “button should be orange, not red”.

You refine stories that are not closed yet. Once the story is closed, it doesn’t get refined anymore, a new story is created or another story is modified.

0

Incremental

so how can one be purely incremental, when each increment will be implemented through the same series of steps?

A dictionary definition will help here:

Increment - NOUN
An increase or addition, especially one of a series on a fixed scale.
‘all sizes from 4–30 mm in 1 mm increments’

In terms of software development, "incremental" is taken to mean "not all in one go". It is the antithesis of "if you build it, they will come", and it's more akin to "build what you need, don't build for possible future versions".

It means that if your application has a FooService class, that there is no expectation that someone will spend X time making the entire FooService class (let's assume it has features A, B, C, and D), at which point it is considered "finished".

Instead, someone working on task 1 might create a FooService class because they need the A feature, so they develop only the A feature. Then, at some later time, someone working on another task finds themselves in need of a B feature, and FooService is the right place to put it. So now, they develop the B feature. At another point in time, someone might need a D feature, so they add it. Maybe after a while, you notice that B's implementation was very simple but because it is now being heavily used, it needs to be more robust, so you upgrade B to B2.

Little steps, one after the other, as opposed to knowing precisely what you're going to build = incremental development.

If an analogy helps: if you're expecting to hold a blueprint of the entire house before you start building it, you're not doing incremental development.

Iterative

the word "iterative" - if the meaning is enahnced to mean "refining and revising work", where does it fit in the classical story development scenario: you develop story B, and once it passes the tests and meets your DoD and Acceptance criteria, it is Done. You added it on the top of the existing baseline. In the mean time, some other stories are finished, ideally Done and hence certainly not meant to be refined or reworked

If you look at a particular feature in your software (let's call it A), that was probably created because of a story.

But when that story is done, that doesn't mean that you will never touch A again. A new story might be written, which brings some requirements that lead to A needing to be changed, extended, or even made obsolete.

some other stories are finished, ideally Done and hence certainly not meant to be refined or reworked

A story should be finished and then not touched again. But the source code that was related to that story doesn't get that same expectation. Just because these lines of code were related to story A doesn't mean that there isn't (or won't be) a story B which also influences what this code does.

Stories are rigid, and will be finished and done with. The code that was written for that story, however, is a living entity which should gradually be changed/updated to keep adapting to the ever-changing needs of the application.

Agile

So, to me it does not seem different from the decades old incremental models (which some claim are examples of Agile models, some say they are not). Just developing features one by one, with early testing and frequent delivery of working increments.

There is a difference between developing one step at a time, and planning development one step at a time. Agile pushes more towards the latter, more so than older models.

The goal is to not be waterfall, i.e. not trying to immediately build a comprehensive solution. People are notoriously bad at guessing the future, and waterfall specifically asks you to guess what you will need for the foreseeable lifetime of your application.

The odds of you spending time and effort building something that you end up not needing, or needing in a different way than you once anticipated, are so high that it makes the initial effort (and mental load that accompanies it) not worth it. At least, according to Agile.

Keep in mind that what I describe here is how Agile sees the world. Not every model works for every scenario.
I've worked on software projects involving major infrastructure safety, and the waterfall model is paramount to ensure that all involved parties are interacting with each other correctly, down to the strictly versioned chipsets in each PLC controller.
It leads to a more rigid process and severely decreased flexibility in development, e.g. you're not allowed to quickly update the chipset's functionality, but when peoples' lives (and billions of dollars in repair costs) are at stake, flexibility is not the most important priority.

12
  • "when peoples' lives (and billions of dollars in repair costs) are at stake, flexibility is not the most important priority" - you seem to be suggesting that Waterfall is still king whenever the stakes are high. But isn't Agile - according to its proponents - also supposed to reduce the risk of massive or serious failures? Or is it more along the lines that Waterfall poses the risk of delivering a working system that is no longer required by the time of delivery - Agile reduces the risk of this, but also increases the risk that the system will contain design faults for lack of planning?
    – Steve
    May 3, 2021 at 9:07
  • @Steve: Agile does not reduce risks, it minimizes the size of the issue when one arises from a software perspective. Agile makes smaller increments than waterfall, so if an update breaks something, it will on average break in a smaller way. However, very much mind the bolded part. In large infrastructure, a small software change can have a big consequence. My favorite example: fire in tunnel. Fans should direct oxygen away. The direction a fan blows in is decided by a single bit. Wrong bit? You've just aimed an industrial, tunnel-sized flamethrower at the incoming firemen.
    – Flater
    May 3, 2021 at 10:33
  • @Steve: For more context, because many different companies are involved in a large infrastructure project, this means that everyone must have their (literal and figurative) wires matched up. To ensure that everyone is on the same version, every document must contain the checksum of the version of the analysis which they were relying on. You literally cannot even correct a typo in the analysis without altering the checksum, and thus requiring an entirely new version. Very slow, not flexible at all, but massive in terms of security and clear communication across many channels over the long term
    – Flater
    May 3, 2021 at 10:37
  • How do you tell the difference between an instance of your "fan control" system that requires Waterfall, and other systems (of lesser importance) that can abide Agile? My experience of most core business systems, is that they are regarded as "fan control systems" that coordinate massive amounts of resources, no part of which can be turned off or operated independently, any disruption to the delicate arrangement of which will haemorrhage cash. Few systems are regarded as low stakes or severable, because making software is far too expensive for things that don't really matter.
    – Steve
    May 3, 2021 at 11:23
  • @Steve: If a mistake can cause death, grievous bodily harm, litigation, bankrupcy-inducing costs for a company (which is "corporate death"), ... they must be safeguarded against. Those safeguard come at the cost of rigidity. In essence it's a matter of business priorities. But this is a very subjective scale. I've seen tons of managers think of their project as "essential", making broad-sweeping rigid frameworks to ensure them, and end up hemorrhaging budgets left, right and center because they overestimate the importance of their project. [..]
    – Flater
    May 3, 2021 at 11:27
0

The iterative process isn't iterative over a single story, it's iterating over a series of sprints. We do a sprint, we evaluate how the sprint worked, we make adjustments, we do another sprint, we evaluate how the sprint worked, we make adjustments, and so on.

From the perspective of the customer, they have a list of product requirements. We break those requirements down into small chunks (epics, stories), we organize these chunks into a backlog of work, and then we iterate over those stories, delivinering incremental functionality with each iterations.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.