I've mostly used the waterfall methodology on my projects, but now I'm expanding my horizons into agile methodologies. From what I've read so far, and maybe I've read the wrong things, agile means small waterfalls. Instead of a big waterfall spread over one or two years, you have small waterfalls that last weeks or maybe a few months at most.
Is my understanding correct or is there more to it than that? What are the agile philosophies?
At a simple level, yes. Simply performing a Waterfall every two weeks does not make you agile, but it is iterative (which is half of agile).
The waterfall model defines phases - requirements, architecture, design, implementation, verification (testing), validation (acceptance testing), and release. In any iterative methodology, you go through each of these phases within every iteration. There might be overlap between them, but you elicit and capture requirements, adopt the architecture and design of the system to allow for implementation, develop the new features or fix the defects, test the new modules, and then present it to the customer for acceptance testing and deployment.
However, there's a lot more to agile than just being iterative and incremental. The tenants of agile are captured in the Manifesto for Agile Software Development. There are four key points made in the Manifesto:
Individuals and interactions over processes and tools
You involve individual people frequently. Many implementations are centered around self-organizing and self-directing teams. Nearly all have frequent interactions with the customer or someone who has voice of the customer. Rather than having a formal set of procedures to follow and tools to use, you let the people working on the project drive how the project gets done to let it get done in the best possible manner.
Working software over comprehensive documentation
In a software project, the primary goal is the delivery of software. However, in some projects, there is wasteful production of documents that add no value. Scott Ambler wrote a good article on Agile/Lean Documentation. It's not about not producing documentation, but about choosing documentation that adds value to your team, future developers, the customer, or the user. Rather than producing documentation that doesn't add value, your software engineers are instead producing software and associated tests.
Customer collaboration over contract negotiation
Rather than defining the terms and timetables and costs up front, it becomes a continuous effort with the customer. For example, you might capture your requirements in the form of user stories and assign them points. After a few iterations, you settle on a velocity (points/iteration) and can determine how many features your team can implement in an iteration. As your customer provides feedback on which features add the most value, they can decide when the project is done at any point. Any number of things can happen with frequent delivery and customer interaction - the requirements have been satisfied and the project concludes into maintenance and eventually end-of-life, the customer finds out that they don't need everything they thought so decides to end the project, the project is failing and the customer sees this early and can cancel it...the list goes on.
Responding to change over following a plan
You don't have a big design or ultimate plan up front and have to perform rework whenever that design or plan has to change. You continually estimate and revise estimates based on the information that you have. You choose your metrics carefully to provide insight into the health of the project and when to make internal changes. You frequently add, remove, and reprioritize requirements with the customer. Ultimately, you understand that change is the only constant.
Being agile means focusing on people and meeting their needs by delivering high-quality, value-adding software quickly. As the needs of the customer change, you adapt to those needs to focus on adding value. There are specific implementations of agile methodologies, but they are all centered on people, timely delivery of working software, and adapting to a rapidly changing environment.
Yes and no - the actual process can be seen as a series of small waterfalls but the difference is that the process evolves and is improved upon from the input of the entire team (dev, qa, business etc), in retrospectives, which should lead to a much tighter unit which is able to react to issues and accurately plan and deliver. I'm grossly over simplifying it here, there is a lot more to it, but I don't think this is a bad starting point.
I would say that is a simplistic way o putting it. Yes, when you get down to it, it is small waterflows, but there is so much more behind it that makes it work. An entire methodology that changes how you approach projects... Not to mention the mindset needed for it.
If you are serious about Agile, here is a good (and long) series of webcasts you might be interested in:
Forget Agile a minute, think what is meant by "waterfall".
There is a requirements phase, in which everyone tries to figure out WHAT problems the end product needs to solve. Folks argue about this for a while, and then they all sign off on a set of requirements. At this point your scope is defined, contracts are signed, and the customer can wander off and wait for you to come up with a product that solves those defined requirements.
Next there is one (or maybe two) design phase(s). The designers (who may or may not be the developers), argue about HOW the system must go together to meet the signed-off requirements. Problems may occur if they don't quite understand a requirement, which may mean they have to go back to the customer, potentially re-opening the requirements phase (and getting another round of sign offs) or at least setting change-management into action. Often, the designers simply make their best guess. They may come up with a logical data model, and lots of UML describing a new system and how it should work. Then they sign off on it.
Now the developers get to actually start coding, based on the signed-off design. Problems may occur if they don't quite understand the design, which may mean they have to go back to the designer, potentially re-opening the design phase (and getting another round of sign offs) or at least setting change-management into action. The designers may in turn realize that the confusion really goes back to the requirements, meaning that they have to re-open requirements discussions, sign-offs, and further change management. Often, the programmers (who have a deadline looming) simply make their best guess. They do what they can to make functional code. Then they release it to testing.
Now the system testing phase sets in. Testers test based on their understanding of the requirements and design, and register what they perceive as defects into a bug tracking/change management system, causing the developers to start developing again, unless they see the issue as a design flaw, which sends it back to design, etc... Eventually the system tests pass and are signed off on.
Finally, the customer comes back and does user acceptance tests on the new system. This is where they decide whether the solution the testers tested, developers developed, and designers designed is actually what they want. If it isn't, you potentially have to revert back to the design phase or even revisit requirements.
The idea behind waterfall is that it is difficult (and very undesirable) to go back once a phase is complete. Different people are generally involved in different phases, so there are multiple hand-offs--each of which introduces a lot of risk for misinterpretation and information loss. There is also a significant gap between when the customers say what they want and when they see what has been built, in which time the real requirements may very well have changed.
Agile methodologies focus on strong communication and cooperation between all the interested parties. The principle "Customer collaboration over contract negotiation" means that you should not have to go through a series of sign-offs and hand-offs, but instead should simply work together with the customer, each iteration determining the requirements for a piece of the puzzle and immediately forming tests, a design, and working code--with all the players communicating as directly as possible (eliminating hand-off costs and risks). The working code is quickly testable by the customer, eliminating time lag risks. All the activities happen in a collaborative swirl, not in a downward flow.
For an excellent overview of what agile methodologies try to do, I'd highly recommend Allistair Cockburn's Agile Software Development: The Cooperative Game.
Yes, that's more or less correct. There has been much written and discussed about how to go about it, but a bunch of little waterfalls is the crux of it.
The specific implementation of how to use a bunch of little waterfalls isn't trivial though. For example, you're not going to switch from making big projects in a couple of years to making a number of small projects. There's still the big project with 1-2 years worth of work to put into it. So you need to split the big project into a number of small projects. That may seem pretty obvious, but it fills pages and pages of books.
Is that correct or is there more to it than that
Both. Yes that is an accurate summation of the concept, but there is a lot of detail being summed up. I mean, pretty much every aspect of planning changes when you are planning for just the next week instead of the next year.
If you look at the static structure (process definition) of an agile project, it does look like many small waterfalls yes. But the goal of an agile project is to get quicker and better feedback.
The agile manifesto highlights some differences between agile and waterfall (as perceived by those who signed).
Really "agile" often means waterfalls of 1-2 days, not weeks. That does not mean you don't follow an overall plan, and that the real release cycles are 1-2 day. But you should try to have a working, tested product every day, and you could release it - in theory - every day.
Scrum, for example, uses sprints of 4 weeks, but a sprint is not just one waterfall (at least, it should not be). You can change priorities every day whenever you see that something does not go the way it was planned at the beginning of the sprint.
