My question is quite simple. How to handle a situation where the team is applying agile methodology in software projects and there so many iterations and change in requirements, that the schedule is highly affected? Plus you have your manager who is always wanting things to get done within schedule and it is something you cannot control?
It sounds like your manager does not really buy into Agile, but rather tries to put an Agile peg into a 'fixed date' mould.
You can run an Agile project with a fixed end-date, but then all parties should understand that you will just deliver whatever features you have completed up to the end date. You can not agree to both a date and a feature set beforehand and then allow the customer to make changes at will without consequences to the final delivery.
One way you could combine an Agile development team with a fixed-date project is to have a strong, in-house, product owner who carefully protects the project scope and keeps the change-request negotiations with the real customer away from the development team. The product owner is then the responsible for avoiding missing the deadline due to scope creep. This assumes that the original schedule was realistic in the first place.
Unfortunately, the situation you find yourself in is probably one of the most common. Agile is gaining adoption in many markets due to buzz, and not because of understanding. The same old problems still exist: desire for fixed scope, fixed date, and fixed cost. If the iron triangle is fixed, no development process is going to save you.
When you encounter the iron triangle, a compromise will need to be made with the client. In your case, your client might be your manager. The client needs to choose from the following (or a balance of all of them):
- Scope adjustment. Drop some features to allow for room for changes. Some changes to already-delivered features are indeed more important to make then building new features that might still be in your backlog. Your Product Owner (or whomever is helping determine priority) can work this out with the client so they don't ask for a change and lose a feature they actually wanted more, or vice versa.
- Schedule adjustment. Allow for a later delivery date, or allow for multiple delivery dates where an initial set of features is delivered on time and then the team delivers a second release with the new scope at a later date. Is it okay to deliver on date and have that change 2 weeks later? This is iterative development, so things should be able to be continuously delivered in many scenarios.
- Budget adjustment. This is probably the hardest to do, as most clients have fixed budgets for projects. If the date is really important (go-to-market plans are in place and missing the date is not an option) and the scope cannot be reduced to fit into the timeline, you can use a request for additional budget to increase the size of your team to allow you to do more within a given iteration.
The most important piece is that the client needs to understand the impact of what they are asking for. When they ask for a change, they are either losing a feature, missing the date, or increasing the cost of the project. Making sure the client understands the impact of their request is paramount, and one of the reasons that you really need somebody playing the role of Product Owner with the client in order to keep the expectations set.
In your case, your team needs a team lead that interfaces with your manager and is having these discussions with them about the impact. Your manager doesn't want you to fail either, but sometimes it is just a lack of understanding somewhere along the chain. It's possible your manager might have somebody coming down on them about the schedule, so giving your manager ammunition to go back up the chain and fight for more time, budget, or scope reductions will help them and you.
If you truly have a fixed schedule for completing a project, you need to use a process that is less agile.
If your timeline is fixed, the requirements (aka features, user stories) also need to be fixed, or at least fixed in size. To meet a fixed schedule, you need to lean more towards "following a plan" rather than "responding to change".
You need to have change management: if someone requests more features, the managers need to decide how to accommodate the request without lengthening the schedule. Jay's answer lays out the scope/schedule/budget trade-offs of traditional project management.
My experience is that there are often not good options for paying money to get more features within a limited timeframe. In that case, the choices come down to
- Reject the new feature
- Cancel another less important feature to make room for the new feature
- Accept a cheaper implementation of certain features. (example: The quarterly sales figures are displayed as text, but not in an interactive 3d barchart)
The problem is that software development is done and managed by people and people (especially management people) don't like to say no to the client because saying no is hard. Oddly clients tend to accept it better than people think they will. In their hearts, they know that new features will take time and cost money, but they have been trained to think they can ask for the moon by people who will work 90 hours a week to accommodate requests that should have been put off until later.
Whether you are doing agile or waterfall is irrelevant, the problem is the person who is communicating with the client doesn't have the guts to tell him the truth. If you ask for new features, they will cost $X more and take X more hours to do and the deadline will move.
The best you can do is push back when it happens with a new cost estimate, a new estimate of the hours to do the work and a new deadline or a suggestion that the new feature be included in the next iteration or a suggestion of what will be dropped to make room for the new feature. This needs to happen in writing and it needs to happen every single time that scope creep happens.
As a developer, when you are asked to add a new feature, you point out the features in this iteration that are not yet started or mostly incomplete (You don't want him to eliminate a completed one or one that is almost ready) and then ask which one he wants you to eliminate to accommodate the new feature or what the new deadline is. Never accept more work without either a change in deadline or other unfinished work being moved to a future iteration. However, this will only work if you get eveyone on board with it. One person accepting this type if scope creep will undermine the rest of you. One time of accepting that simple-to-make change will undermine your chances of pushing a complex change further down the road. You and your co-workers need to be disciplined about this.