What do you need to succeed with Agile?

Question

Agile adoption can fail in some organizations, I even worked for a company where waterfall was the only (the true) way but only because they tried Agile on a project and failed.

When I asked the people who still remembered that (I was a junior) I was shut down hard, like I was reminding them a bad nightmare that really happened.

I don't know why the project failed. There are resources found on the web why Agile fails is some companies but the reason are mostly economic. So I thought I ask here fore some feedback.

What are the reasons Agile adoption fails in some organizations? Or, another way of putting it.. What do you need to succeed with Agile?

Answer 1

You need management, clients and developers each to understand and support the Agile way of thinking and the Agile methods. In more detail:

• Management must be comfortable with hearing the truth, as opposed to what they traditionally expect to hear (i.e. "yes, the project is on track" for 11 months... then suddenly "oops, we need to delay the deadline by a few weeks... erm, months... erm..." in the very end). They must also be comfortable with letting each party do (and take responsibility for) their job. Most prominently, that the developers are to make technical decisions and estimates. So no tight controlling and micro-management.
• Clients must understand that Agile requires their deep and continuous involvement in the development process, not just "making the order", then picking up the delivery some months later. They also must be comfortable by the lack of a Big Fixed Requirement Specification and the apparent lack of commitment from the development team to deliver what they were initially requested for.
• Developers must be comfortable with taking responsibility, making decisions, communicating openly and working as a team. I.e. no "code ownership", no "lonely cowboys" and no fruitless blaming of others for problems which can be solved by the team itself.

In my experience, it is the first point which most often lies behind failed Agile projects, but the other two can also cause problems.

Update

By "management" I mean not only the direct project manager, but higher levels too. As @Michael very rightly noted, some more or less external parties (e.g. QA or an external task assigner) can also affect the sucess/failure of Agile projects, but I suppose that can only be possible if their respective leader lets them, and/or if responsibilities and lines of command aren't clear within the organization.

Answer 2

You need:

• People who are willing to be very open and honest. Visibility is everything because you need trust across all sorts of layer-boundaries.
• Customers and managers who really want to hear the truth.
• People who are willing and able to redefine their roles to suit what's needed right now.
• Freedom to change processes that aren't working right now.
• People who are willing to admit mistakes and reverse them.
• The ability to throw together build and test environments at will. (This is more important and more expensive than people give it credit.)
• As many whiteboards as you can fill your walls with.

Seems so simple, but a lot of these are big asks in this industry.

Answer 3

An agile project will most often fail when a key player refuses to be agile, or (worse) when someone is not honestly interested in the project's success or outright sabotages it. The latter can kill any project (like a number of other things), but agile processes give people more flexibility, and that includes the flexibility to play destructive politics.

Examples:

• QA insists that customers cannot see the software unless it has passed a month-long manual test cycle
• Management imposes unrealistic deadlines
• Customer refuses to prioritize requirements ("they all have the highest priority!")
• Someone who's not an immediate stakeholder but has political clout keeps assigning lengthy, unrelated tasks to a key team member, and the project manager can't prevent this.

Answer 4

I can only give my advice from my own personal experience.

One employer I had totally failed at Agile. Work was done on an ad-hoc basis, testing was non-existent, and requirements were documented in emails and meeting minutes. The only development method used was anarchy, or 'fire-and-forget coding'. Implementing some kind of software engineering method would have been impossible as developers were too overworked to set up some kind of story-tracking project management software.

At another employer, our team had a heroic member who desperately tried to establish some Agile best practices - we had a Kanban board, issue tracking, we used TDD and BDD (while not Agile in themselves, they tend to be present in Agile groups). Unfortunately, we lacked things like story points, estimation sessions, capacity planning, burn-down charts, velocity graphs - the kind of useful Agile project-management stuff that allows work to flow through smoothly. As a classic symptom of Agile going wrong, when our Kanban board got too full, we bought a bigger board :/

The place I'm currently in uses story points as a way of planning capacity with two-week iterations, TDD, daily standups, iteration-by-iteration timeboxed retrospectives and pair programming on most things. This is as a result of total management buy-in and client education.

It think that in order for Agile to succeed at a company, you need the following things:

• Project managers who understand Agile and who will use the tools appropriately.
• Developers who understand Agile, who are open and honest, with the discipline Agile requires
• Buy-in from the client. They need to recognise the benefits of Agile and be willing to listen to advice from their developers with regards to what can be developed in a given time-frame.

EDIT: It's also vital to make sure you have a good understanding of -why- things like daily stand-ups and short iterations are useful.

Answer 5

My experiences with Agile failure have had nothing to do with economics but with corporate/departmental/personal politics.

On a personal level, there are simply some people who’s personalities will clash. Forcing them into an Agile team, or even worse a paired programming team, will escalate their dislike of each other to a boiling point. This can get very nasty, very quickly and result in things like acts of sabotage worthy of a reality show, turning scrum meetings into a circular firing squad of blame or even worse.

Above that, there’s development management. I’ve seen this go wrong in two different ways.

First is ‘cargo cult Agile’ where the manager insists upon following the manifesto and whatever class/book/website they read exactly without understanding the reason why and when to use them and when to improvise. It’s as if the Agile manager is waiting for the magic to happen because they’re following the spell exactly. This procrustean implementation of Agile can result in a number of problems that will lead to project failure.

The other is ‘Agile In Name Only’ where the terminology like sprints and scrum are used but are really just labels over old practices like micro-management, dishonesty going both up and down the chain of command, lengthy useless status meetings and other such stuff. Projects fail just like they used to but now Agile can be blamed for it rather than poor management.

Above that is a lack of buy-in by the client/customer of the project. These people will have their own departmental priorities and can be resistant to working with a development team unless it’s made clear that it’s an essential part of their job by their management. This can be made worse by departmental politics or corporate policies. For example, both operations and marketing have input into a project and your team ends up spinning their wheels since the two sides cannot agree on anything. Another example is when corporate policy on time management and billing causes conflicts. I’ve actually found that external customers were easier to deal with than internal ones. They liked the attention they got from the process and knew they were getting their money’s worth.

Answer 6

IMO "Agile" fails when there isn't a wholesale buy-in of the practices. What I mean is that Agile relies on the "customer" (typically another department or manager) understanding that:

1. They don't know 100% what they want the software to do, even if they think they do
2. They have no idea how long it will take to complete, even if they think they do
3. They will be told how long it will take and must accept it or reduce features to meet a deadline
4. They will have to choose features each iteration and will not get the full, 100% complete application in one shot.

All of those are very difficult things for most managers to swallow, and IMO the #1 reason why it's hard to do Agile - managers are used to saying "It will be done by x date" and having it "Magically" done by that date (after developers put in 80 hour weeks) and it's a 180 to realize that the development team is going to tell you that this project will be done in 3 months, and the only choice you have is to accept it or reduce the requirements to get it done sooner.

Second, there has to be trust in the development team. Going hand in hand with #1 above, very few managers actually seem to trust the opinion of people hired as experts; if a developer says a project will take x amount of time, and x is more than management thinks it will take, it's never a case of "I don't know how to write software, so the developer is probably right" it's more "Those good-for-nothing developers want to goof off at work so they said it will take 3 months".

Third, your development team needs to be behind Agile; that means not cutting corners and not ignoring constant feedback and questions of "Is this right? When x happens what should Y do?". Even if you don't follow TDD or Pair Programming, your development team needs to be competent enough to follow the agile processes (e.g. sprints, iterations). This involves having a lead/manager who can properly estimate tasks and understands that you don't need to make every feature a priority, it's OK to have a backlog of work, and you shouldn't overwork people.