43

I have worked with Scrum in various IT companies on software development projects. This framework has become the standard in several companies, but I have not yet managed to make friends with it because it causes me a lot of stress.

Estimates

We estimated the backlog items in hours. I often struggled with the fact that I found it very difficult to give estimates. This was due to

  • the complexity of the task
  • the associated dependencies / possible side effects
  • the own experience (e.g., when I had to implement something that I had never done before)
  • uncertainty / not completely precise requirements
  • existing legacy code
  • other technical difficulties / dependencies in the code

When my estimates of the backlog items were considered too high by other developers and/or the product owner, there were sometimes tedious and exhausting discussions. I felt like I had to justify my estimates, which as you can imagine is quite unpleasant. In the end, it came to a negotiation about the number of hours, which was entered as an estimate for the backlog item, in which I felt compelled to adjust my estimate downwards so as not to step out of line or come across as too pessimistic or even incompetent.

Burndown Chart

The progress of the project team is visualized almost in real time (or at the end of the working day) using the burndown chart in the sprint backlog.

This means that you can immediately see "mercilessly" whether you are on track or behind schedule compared to the estimated time. I also find this point unpleasant, as it makes me feel under pressure in my daily work. As soon as there is an unforeseen event (e.g., something that was not considered in the estimate), this has an immediate effect on the burndown.

When it took me more time than estimated for the implementation, then I always had a bad conscience. However, having taken more time than estimated doesn’t explain

  • whether I had worked badly/slowly
  • or I had only estimated badly/inaccurately
  • or whether the obstacles/unforeseen events were not foreseeable at all
    • either because of my own level of knowledge
    • or the problem in question could not have been foreseen at all

This in turn leads to discussions / justifications in retrospectives. And as soon as the sprint is over, the whole game starts again with a new sprint.

Questions

  • Do I have to accept that Scrum is annoying/stressful based on the above description, or is there a way to deal with it differently which might lead to a different experience?
  • How can I overcome the bad conscience when I’m wrong with an estimate or when I don’t manage to finish the planned tasks by the end of the sprint? I have experienced other developers who did not care at all (their statement was that "it was just an estimate / a guess"). However, their manager did not confront them directly with it.

As I understand it, Scrum should have been developed to simplify the lives of developers. However, due to the points mentioned above, I have the feeling that the exact opposite is happening.

Are there any misinterpretations of Scrum on my part? It is always important to me to do a good job, but due to the circumstances described above, Scrum has robbed me of energy that I would rather have invested in the actual work.

19
  • 28
    This might be a better fit for the project management forum but my personal opinion and experience tells me that scrum has become somewhat disconnected from core agile principles. For example, I think conflating estimates and commitments leads to poor outcomes.
    – JimmyJames
    Nov 21, 2023 at 21:41
  • 29
    What your company is doing isn't Scrum, but ScrumDoneWrong. They use the buzzwords and bastardized versions of the practices but don't understand the intent and reasoning behind them. Nov 21, 2023 at 22:51
  • 16
    "As I understand it, SCRUM should have been developed to simplify the lives of developers." Haha, no. Scrum was presented, by smart people, to management that felt out of control over software development, in order to give that management the feeling they actually CAN control software development. Because that is a marketable idea. Developers were never a factor, they only got neutered and were made irrelevant in the process, which is what you are experiencing. Nov 22, 2023 at 7:44
  • 11
    Software estimates as a single point value are basically meaningless. An estimate is a "most likely" central point, and a range from "best case" to "worst case" around that "most likely" point. Without that range, you have no encoding of the risk/uncertainty in a task. A task that's "most likely 3 days, 1-3 days" is very different from a task that's "most likely 3 days, 2 - 25 days". Nov 22, 2023 at 8:02
  • 15
    Maybe it should be called a "burnout chart" rather than a burndown chart.
    – Stef
    Nov 22, 2023 at 9:18

11 Answers 11

50

Warning: this answer doesn't provide a definite solution, it addresses why there isn't one. Some people don't like that, so don't read this long answer if you're one of them.

At the end of the day, these systems are there to manage the workload and meet the company's expectations. Therefore, any given answer depends on what the company expects and what is actually at risk of happening without the system. Scrum is really good in certain cases, but it can seem like overengineering or micromanaging in other cases. You have to consider the context here.

I'll use my current context as an example.

Anti-scrum

My company delivers a product, the high quality of which ensures that we retain customer loyalty. Tech budgets are not as constrained as they usually are, but the company's expectations of high quality are significantly more sensitive. This leads to certain behaviors in my team:

  • We don't get caught up on deadlines, we'd rather miss them and deliver a quality product as opposed to pushing something out before it's ready.
  • We sometimes explore and performance test several avenues, which is not something we can pre-emptively analyze, we have to develop multiple solutions and decide whether we're happy enough with them or want to investigate further.
  • We are constantly available for any production outages or support issues, which means that our development work sometimes takes an unscheduled back seat.
  • It has been known to happen that product priorities change, and work can go in the freezer because other work has turned out to be more important (I want to point out here that there is no friction betweens devs and PO here, this is done with mutual understanding from both sides)

The final conclusion here is that scrum is not right for us. We cannot consistently estimate our work, nor can we lock down our focus for any given sprint, nor are we guaranteed to follow through with prior commitments.

We still use scrum technically (because the company at large does rely on it), but we've clearly adopted a kanban approach. We barely estimate tasks, we don't commit things to a sprint, and we carry ongoing work over to the next sprint without any real scrutiny.

Pro-scrum

However, there is another team at our company where scrum is actually helpful. They are different from my team in the sense that their development is driven by marketing, not product platform development. This changes the situation for them:

  • Deadlines must be met, since there is a rigorous coordination between advertisements, promotional materials, and availability of the related features.
  • Work is rigorously defined. There are no open ended "let's see what we can do" tasks. The exact expectations are defined and shall not be deviated from (without PO approval, which rarely happens).
  • Support issues are handled by a separate team, not the developers. I'm skipping the precise explanation why (it would reveal too much) but suffice to say that it makes a lot of sense that those devs don't have their hands on those support tools.

Because of this, scrum makes a lot more sense here. Everything needs to be estimated to within an inch of its life, because the company expects that its expectations will be met. The team has to set out a sprint goal and commit to it, because the marketing team relies on that commitment to synchronize the work they do (e.g. hiring advertising and getting promotional material produced).

When to use scrum?

I want to focus back on my team here. Part of what makes us able to do kanban and keep both the PO and devs happy is that we have a mutual respect, understanding and team spirit. If we were to have an adversarial attitude, e.g. if the PO tried to constantly push up our deadlines or argue that something should be done faster, we would not be able to do our work without scrum.

In this alternate scenario, I would push for us to actually estimate our work, because estimations then become a necessary tool to justify the amount of time that a certain task will take.

This has further repercussions, because if the PO were to now use these estimations as near-deadlines, we would be inclined to pad our estimates further to account for possible distracts.
We would also refuse to allow anyone to sidetrack us from our sprint commitment, because we have a deadline to meet. Support issues would hang around on the backlog much longer before they would even get looked at, which would degrade the customer experience significantly (who wants to wait around for two weeks to even get a first response?)

So, in my developer-biased opinion, scrum is a way to hard-define expectations in cases where the company/PO dictates the workings of their tech department. This isn't always a bad thing per se (e.g. if the company's main focus was not on technology). However, in my team's case, the company specifically prides itself on its technological excellence, which makes us favor an approach where the tech department has the freedom to incubate a feature for as long as they deem necessary, which is why scrum activities are a relatively futile exercise for us.


This (finally) brings me to address the scenario you find yourself in.

Your scenario

I felt like I had to justify my estimates, which as you can imagine is quite unpleasant.

Some justification is warranted, but there's a point where it becomes almost persecutorial rather than an open discussion on technical opinions. Based on other things you've mentioned, I'm going to assume it actually became persecutorial.

In the end, it came to a negotiation about the number of hours

It's interesting that you call it a negotiation, because what this clearly reveals is that the tech department does not have authority here. You were asked for a technical opinion, provided one, and then were overridden by a non-technical person.

The key takeaway here is that the development flow is being dictated by someone other than the technical department, which means that your system should (correctly) model that. This actually justifies scrum as a valid methodology, as there is clearly a strict need for well-defined commitments with deadlines.

in which I felt compelled to adjust my estimate downwards so as not to step out of line or come across as too pessimistic or even incompetent.

The main takeaway for me here is not that scrum is wrong for your company, but rather that there is a toxic attitude from your PO/manager that is being used to apply additional pressure to the developer(s).

Even if the driving force comes from a non-technical source, that does not mean that they can't respect your technical opinion. I've worked with fantastic non-technical manager who would not dispute the estimates they were given. At best, they'd either decide to simply not schedule the work because there were more pressing concerns, or ask what we would need to change about the requirements so that the estimate could be lowered to a certain target value.

This means that you can immediately see "mercilessly" whether you are on track or behind schedule compared to the estimated time. I also find this point unpleasant, as it makes me feel under pressure in my daily work. As soon as there is an unforeseen event (e.g., something that was not considered in the estimate), this has an immediate effect on the burndown.

I'm going to err on the side here of saying that the burndown chart is exactly supposed to show you these effects in the short term. That's what it's designed to do.

From your explanation, I'm inferring that you are immediately taking this as a reflection of your work quality, which is simply not a good way of tackling this, and likely a large contributor to your stress. I would offer the suggestion that you can look at this from a more distant perspective, whereby the burndown chart will fluctuate and you shouldn't worry about it because if it's unforeseen then you could by definition not have planned for it, and if it's not foreseen and the burndown chart consistently shows a problem, then maybe there is actually a problem worth addressing. Neither should be a source of negative emotions, the chart is there to provide metrics to assist your work.

However, if my assumption about toxic workplace attitude is correct, that would explain why this is a source of stress for you, i.e. because your management is actively using it to negatively judge you. If so, then I just want to point out that management is the source of your stress, not the burndown chart that they happen to be used to put this pressure on you.

I also want to be very clear here, I'm assuming toxic workplace behavior based on how you describe your response to these things happening. I cannot judge whether someone is genuinely putting pressure on you or whether you're fearing that kind of pressure and pre-emptively defending against it.

However, with what you've said, my guess here is that there is pressure on you that shouldn't be. Your issue isn't with the specific approach or tooling, but rather with how it's being used to put pressure on the developer (regardless of who is applying that pressure).

11
  • 4
    This is a good read, but first "mutual respect, understanding and team spirit" - you seem to imply that scrum is for those companies without these things. No respect, no understanding, no team spirit? Scrum is for you! Secondly, in your description of the marketing developers, how does their "scrum" differ from "waterfall" (or indeed, just any ordinary activity that is planned, requires resources to be coordinated, and assumed to have a schedule). (1/2)
    – Steve
    Nov 22, 2023 at 11:03
  • 1
    @Steve: You've fallen prey to the fallacy of the inverse. I didn't say companies that do scrum have no [...] in the workplace, I said that [...] contributes to us being able to not do scrum and instead leave things more open ended in terms of estimates and deadlines. What I am saying, is that if there's no implicit trust (i.e. trusting that the devs will work in a timely and reasonable manner even without putting an estimate to paper), then scrum enforces a written agreement so as to provide that otherwise absent trust in the developers' efficiency.
    – Flater
    Nov 22, 2023 at 11:12
  • 2
    @Steve: Waterfall (or lack thereof) plays out on a scale that requires more than a single sprint. To put it differently, even in an agile system individual sprints are relatively locked down (at the start of the sprint) and waterfall-ish. What agile doesn't do is already plan the subsequent sprints at the same time, instead relying on reassessment inbetween sprints.
    – Flater
    Nov 22, 2023 at 11:15
  • 1
    @Steve Scrum is a solution in scenarios where externals (i.e. not the devs) need assurances with regards to the delivery of the work (whether that be related to budget, deadline, ... is a different discussion) - a more cynical way of saying the same thing is that this provides "proof" that the devs didn't stick to their promise and missed the deliverable (as opposed to the externals themselves). This is not always bad, I'm not implying that devs should always rule the roost. This is highly contextual based on the company's structure and goals.
    – Flater
    Nov 22, 2023 at 11:18
  • 6
    @Steve: It's pretty rich for you to claim that trust and respect is an irreducible minimum, while at the same time accusing non-technical management of engaging in practices that amount to (from your comment above) a charade, a bait and switch, using a cudgel, (from your own answer) intentionally sweating developers and exhausting them, turning the screw, and domination. Again, I'm not saying this doesn't happen, but so does inefficient development that never delivers. You berate the other side for painting your side with too wide of a negative brush while doing the exact same thing to them.
    – Flater
    Nov 22, 2023 at 22:26
36

Let me say something: I love Scrum if done right. What you describe sounds really stressful though and I would hate it, too.

There seem to be two big problems that you have:

You estimate tasks in hours. And you feel bad when you don't make that estimate.

One is an external factor. Estimating tasks of a team in hours is just an objective failure by whoever asked you to do this. You cannot. Not because of any method or agile principles or because some book or another had a better way, but simply because it goes against reality.

As an example, lets say we have a small team of four. One is a 20 year veteran who build the companies current system from the ground up. They know it inside out. Then you have two rather normal, mid level experience developers, one has been working on this system for two years, one was just hired last month from a competitor. And we have an apprentice in their second year.

There is no way that those 4 people could possibly come to a conclusion on how many hours any development might take. Because even with perfect information and perfect motivation and perfect work, it simply takes them different amounts of hours. And it is absolutely pointless to have a "negotiation" about it. The apprentice cannot be negotiated to work faster. The new developer cannot be negotiated to magically gain insight into the system the same way the developer already knowing the system has. It just doesn't work that way. Even in the best case of omniscience, they will arrive at wildly different numbers, that are all true.

I will not tell you what the "correct" way is to do that. But there are multiple ways to handle this and they all try to measure the size of the task. In fibonnacci numbers, or t-shirt sizes or their own system. But they estimate the task. A task can be estimated by all and all can come to the same result. The senior, the apprentice, the developers, all can agree that changing a button text is probably a "small" task and changing the reporting algorithm for quarterly numbers is probably a "big" task.

And it is blindingly obvious, that the 20 year veteran will finish either of those tasks faster than the apprentice.

But the point of "negotiation" here is that we don't have the absolute clarity sometimes, so if the apprentice says "oh, this task seems small", the veteran can chime in and say "oh, but you need to consider this and that" and then the apprentice will maybe say "oh, I didn't know that, with this information, it indeed looks like a big task". Negotiation about the size of the task is always neutral and always about the fact that different people have different knowledge and to estimate we should bring all the knowledge to the table.

That said, lets assume you drop this abomination of estimating in hours.

Your problem of feeling bad when you don't live up to an estimation immediately becomes less severe. What do you measure "small task" against? Who would be telling you how many hours you can or should work on it? At which point you are to feel bad about it?

An important thing first though: you are working against an estimate. It is perfectly normal to not make that. You are either done faster or slower, that is the nature of estimates. If your company does not throw you a party with a raise every time you are faster than estimated, don't feel bad about being slower either. It's just the nature of imperfect work and missing information. If you did your best, just have the self-esteem to be happy with it. Most of what I read is "your consicence" and no process can lift that. You need to work on your conscience being more forgiving.

I am not sure why not making ticket estimates lead to disussions in retrospectives. Are those productive discussions, are they leading to improved work and either better estimates or better work? I assume not. So stop them.

May I ask if you have a Scrum Master? Because in Scrum there is a specific person whose whole job it is to make sure you are doing Scrum, and who should scream at the top of their lungs at the thought of estimating in hours or discussing personal performance in the retrospective.

You said you have worked in multiple IT companies this way... maybe it is time to get smarter in interviewing. Let the company describe their Scrum process to you in the interview. Don't sign at a company that does things you are stressed about.

3
  • 10
    "And it is blindingly obvious, that the 20 year veteran will finish either of those tasks faster than the apprentice." Not necessarily: it is entirely possible that the apprentice will be able to knock out a (barely) working solution in less time than the veteran, because the veteran is aware of the dragons and how to work around them (or needing to spend some time to vanquish them). Nov 23, 2023 at 12:21
  • 5
    @MarkRotteveel Or just blind luck. If a component happened to have zero actual dependencies, the junior will work on the task and blindly blast through it, while the senior would verify that it truly has no side-effects. Strictly speaking, the junior would've made a mistake but he got lucky, yet the effect is that he "finished" the task faster by being ignorant.
    – Nelson
    Nov 24, 2023 at 3:05
  • Well, I consider a task "finished" when it was done according to the DoD, which for any team I have worked in included a code review and testing by the team. Obviously, in any broken team with broken DoDs, producing broken code, it is anybodys guess who finishes first. But you can produce a crappy result with any method, as long as you do it wrong, so I won't consider that in my answers.
    – nvoigt
    Nov 24, 2023 at 12:15
23

As I understand it, Scrum should have been developed to simplify the lives of developers.

No, that's what agile software development was created for. Scrum was created to ensure that management retains roughly the same level of control it had before agile software development existed, and that it can continue to work pretty much as before without having to adapt too much, and still be able to sell the idea of "we're agile now" to developers without having to become truly agile itself.

Agile development itself does not require most of the things that Scrum introduces. And I don't see how anything that Scrum introduces really makes life easier for developers compared to agile development without Scrum. I only see how it makes life easier for management and those who want to have more control over the developers and the development process, as if it was development prior to agile.

However, due to the points mentioned above, I have the feeling that the exact opposite is happening.

This is why so many companies love Scrum, because Scrum allows them to misbehave just like they did before agile development. Scrum itself does nothing to prevent this misbehavior, it just hides it behind new names (roles) and on top of that makes it easier for management to blame every failure on the developers (they made wrong estimates, their performance was too low according to the burndown chart, they didn't finish a feature by the deadline), while in reality it is mostly management mistakes and false promises to customers. And because developers have been told that they are now in control and everything is their responsibility (neither of which is remotely true), they often believe it's their fault and blame themselves.

Don't get me wrong, you can use Scrum without abusing it in this way, but many companies only use Scrum so that they can claim to be "fully agile", when in reality they are just hiding the fact that they are not. And if you just wanted to be truly agile, you wouldn't need Scrum at all. Scrum just makes everything more complicated than it needs to be and only adds extra overhead to the development task, which always has a negative impact on performance, as every second you spend on following the scrum protocol beyond what agile itself dictates, is a second you don't spend on agile software development itself.

The agile manifesto is basically:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

But Scrum is a process and deadlines after burndown charts are contract negotiations. The whole idea of being agile is, well, being agile. Agile in the sense of being flexible and fast and getting things done instead of following static rules... static rules like a whole set of rules created by Scrum. It's management that wants more rules, because rules mean control.

If the development team delivered great products without management, which most development teams certainly could, 9 out of 10 managers would no longer have a justification for their job. This is why Scrum creates an additional role for these people, not because the role is necessary or because the agile manifesto requires the existence of such a role at all. Rather, this role adds a new barrier between developers and customers, which the agile manifesto actually wanted to tear down, as it was convinced that the closer the developers and customers get together, the better the product will be and the better it will ultimately meet customer needs.

As for time estimations, I go with Uncle Bob ("Robert C. Martin", author of "Clean Code: A Handbook of Agile Software Craftsmanship" and inventor of the SOLID principle), who basically says that you cannot estimate software. You simply cannot know how long it will take to create something you've never created before, because following the DRY principle, you should only ever write new code and otherwise re-use what's already there.

So according to him, an estimation for work should just a value between 0 and 5. Why? Because programmers are good at doing relative estimations but they are bad at doing absolute ones. What does that number mean? Nothing! 4 is more work than 3 and no, it's not twice as much work as 2. A higher number means more work, a lower number means less, but these numbers have no relation to hours, days or weeks and they have no relation to each other except that higher means more and lower means less. In another talk he said that 0 can be a valid option and would mean "no work at all" for cases where nothing needs to be done if you are happy with a particular implementation that really requires no work, otherwise it must be at least 1.

When forced to give a time estimate for a concrete implementation task, he always gives three figures: Best case, worst case and nominal case, whereby the range between best and worst can be large, which expresses the realistic uncertainty. Whether such an estimate is useful is another question, I suspect not, but that's a management problem and not your problem, unless you are hired as manager and also paid as a manager.

And the point of a burndown chart is not to monitor individual performance (again, any kind of live monitoring goes directly against the agile manifesto), but to see how many points the team is able to deliver in a sprint, what gives those points actual meaning, and to see if certain changes in the team or work environment make the team more effective or less effective. The system shall never impose pressure on anyone, as it shall not measure individual performance, but team performance. And if a team can only deliver x points a week, then it can only deliver x points and that is not problem, no matter how low x is, this is information the management has to deal with when they decide about deadlines or make product promises to customers.

The whole idea of the agile approach is to trust the developers. To trust them to do a good job and work as much and as fast as possible without sacrificing the quality of the code (= the product). Forcing developers to work faster and sacrifice quality to meet arbitrary deadlines is exactly the problem the agile manifesto wanted to eliminate! If a customer needs the product faster, they need to cut features or simplify parts of the product in consultation with the developers - that's what is meant by "customer collaboration over contract negotiation". The basic idea is that the customer can decide what features need to be in the product, or they can decide when the product needs to be ready, but they can't expect the list of features they have come up with to be ready on the arbitrary set date they want, that's not how things work and management must also not make promises like that.

If you rush code, you create bugs, and bugs cause problems, and problems slow down development, so you end up creating less and less while trying to create more, which is moronic. Code builds on other code, and if the code it builds on is already buggy, the code it builds on won't work right either, and eventually nothing works right, and everyone spends days to weeks fixing bugs instead of implementing new features. You can choose from several alternatives the one that is quickest to implement when time is of the essence, but you can't expect developers to write bad code on purpose. If I want to make something out of steel and I get behind because the welding takes too much time, I can redesign the product so that less steel needs to be welded, but I can't go to the welders and tell them, "Just make worse welds to speed things up," because then they wouldn't have done their job right, and in the end a few of those welds might break and the whole thing falls apart, and then you have no product at all. Welders don't ever intentionally make bad welds, that's totally against their work ethics. If you need to speed things up, you need more welders or must reduce the number of welds to be done, that's how this works. Or as Uncle Bob once said: "We will not ship shit!"

Here's Uncle Bob himself speaking on time estimates:
https://youtu.be/Qjywrq2gM8o?t=3346

And here he speaks about point estimates:
https://youtu.be/l-gF0vDhJVI?t=3473

The entire presentation he gave is very valuable. It's not about agility in general, agility is only mentioned at the very end. Lots of interesting topics for programmers, and even if you don't agree with everything, you can change your perspective on certain things. But even more important than for developers is the talk for managers, because mismanagement is much more often the main problem in software development than what the developers do. You can have the best team with the best developers, if it's led by a lousy manager, you're just going to get lousy results.

And that seems to be your actually problem:

Your managers apparently don't know how to do good management, they haven't understood agile development, they hide behind Scrum to repeat all the classic software management mistakes and to put the blame on the developers for their own management failures. And that's why Scrum sucks for you, because the scrum concept can be misused as it does not setup any rules that would prevent such a misuse and IMHO this can only have one reason: The possibility of such a misuse was intentionally kept open for those managers who want to have agile development without really having agile development and who are not willing to give up any control to the team as they should and who just want to hide the fact, that above the developer team nothing has really changed and all past mistakes are repeated. If those managers read a book about true agile development, they freak out but then they read a book about Scrum and think "Oh, I can make this concept work without having to change a lot, without having to change myself or the way I manage those software guys". And that's why Scrum is the popular among managers and thus used by so many companies.

Update

Here's another video from Uncle Bob I found that's entirely focused on time estimations and why managers simply have unrealistic expectations about it.

https://youtu.be/eisuQefYw_o

Quote: "There is a fundamental truth to work breakdown structure (WBS) estimation: The only way to estimate using a work breakdown structure, to really accurately get the number right, is to implement the project. [...] So you have to make sure that managers understand that there is an extremely high cost involved with refining the estimate."

Product owners always want to persuade you to make false promises because then you either punish yourself and do everything possible to keep the promise, or you fail, but then all the blame lies with you. That alone is a violation of scrum protocol. Product owners have no say at all when it comes to estimates. The developers make estimates and don't have to justify them to the product owner. The product owner only has a say when it comes to picking which stories go into the next sprint.

If the product owner knew better, he would implement the software himself instead of hiring developers to do it, because it would be much cheaper and apparently he can do it much faster than trained professionals, right? The fact that he's paying you to do it already disproves this claim, and if he can't do it, he can't know how long it will take you. All he wants from you is a false promise, so the blame lies with you and not him, after all most product owners have people above them that they have to justify themselves to and they want you to be their scapegoat when things go wrong.

Show them that you can meet your estimates. If you estimate 5 days and they push you to estimate 3 days, but it ends up taking 5 days, point out that your original estimate was actually correct. There is no point in improving the estimate if it ends up taking the same amount of time as the original estimate. Managers need honest estimates, not good-looking estimates. Good-looking estimates are sandcastles over the rainbow and totally worthless. They look good on paper but they have no real business value, as you cannot use them to estimate how long the project realistically will take or what it will cost or when you might be able to ship it. They just look good, but that's all they do.

Update

I like to add two more talks here from Allen Holub, as they are straight on the point. 1990 Allen Holub wrote the book Compiler Design in C, still considered a reference book as of today, and some may also know his book Taming Java Threads.

In his first talk, he basically explains why agile doesn't work for most companies and why Scrum is not agile (he covers a lot of the same arguments I've already covered above, but I didn't even know his talk, when I wrote my reply; especially he agrees that the whole point of Scrum is to fake agility and prevents companies from truly becoming agile as become agile means change and companies don't want to change their existing structures):
https://youtu.be/vSnCeJEka_s

In his second talk, he explains why time estimations in agile software development are totally nonsense and themselves violate the agile concept (and he also confirms that if your company is truly agile, the first thing you want to get rid of are most of your managers as you won't need most managers anymore as they cannot perform any meaningful work in an agile company):
https://youtu.be/QVBlnCTu9Ms

0
7

Scrum doesn't have you estimate tasks.

Scrum has you commit to turning something in within the sprint time box.

That's not estimating.

Estimates are lies told to get competitive jobs. Don't ask coders for estimates. We are notoriously optimistic with them. Ask them if this task looks about as tough as that last task. That's what points are.

If you let this degenerate into estimating you're losing the fundamental thing agile gave us: a way to show our incomplete work as we go and get early feedback on it.

Lose that and you've just dressed up waterfall with new buzzwords.

If you find yourself on a dysfunctional team that assigns you a task and gives you little input into the tasks points then the feedback to give before starting is how confident you are in succeeding by the end of the sprint. If you can break that up into smaller tasks, do. Even if they wont let you. Report your plan as early as you can. Complete what you can.

Trying to do more than you can is always stressful. The sooner you push back the better. Better for you and the team.

31
  • 3
    @nvoigt the commitment you make is to turn in something. That something could be failing. But you submit it and let the progress get evaluated. Otherwise you're just shoving waterfall in a timebox. Nov 22, 2023 at 11:38
  • 3
    This is the nominally correct answer: Scrum as defined in the 2020 Guide doesn't have estimation, only commitment to a Sprint Goal. This kind of Scrum is great for capable teams in high-trust environments that do great work, but struggle with delivering Value. Scrum helps to start getting a small Goal definitely Done within the timebox. Eventually more gets Done: (a) growing experience with estimation/forecasting, (b) increasing visibility (and Backlog prioritization) of the team's other work that has Value, (c) peace for uninterrupted work. Of course, actual power dynamics work against this.
    – amon
    Nov 22, 2023 at 15:14
  • 5
    How do you commit to doing something in a time box without estimating (even just in your head) how long it will take? If you don't have some idea how long a task will take to complete, isn't it irresponsible to promise to complete it within an arbitrary time frame?
    – JimmyJames
    Nov 22, 2023 at 16:14
  • 2
    @candied_orange What you are saying sounds reasonable but I don't think that's how most scrum masters understand it. They want people to commit to completing a given deliverable in the sprint. If that means burning the candle at both ends, that's the developer's problem. There's this nasty tone to it that reminds me of the worst kind of managers I've had the displeasure of reporting to. When I would explain the factors that were out of my control which prevented me from succeeding, they would tell me I "wasn't being accountable" and were otherwise useless.
    – JimmyJames
    Nov 22, 2023 at 16:42
  • 3
    @candied_orange "in my experience most scrum masters don't do scrum" 100%! That's why so many people hate "scrum" and likely why the OP is asking about this. When I read about scrum, I generally like what I see. When I see it in practice, it's mostly just cargo-cult ceremonies around a non-agile approach.
    – JimmyJames
    Nov 22, 2023 at 17:47
6

The problem with estimates is that companies need them to give customers a price for whatever work is being considered, but they are hard to produce.

To reduce your stress levels I would suggest the following:

  1. Get an idea of the expected deadline for the entire project. This will give you a baseline for each task estimate of deadline/number of tasks.

  2. Estimate tasks in your mind as small, medium, big and then convert that to some standard number of hours. eg small=1 day, med= 3 days, big = 2 weeks

  3. If questioned about your estimate being too long, roll up several related tasks and make a estimate to do them all together, then divide by the number of tasks. Often smaller tasks each get rounded up and then the total comes out as unexpectedly large. Getting a rough estimate for a larger piece of work can be easier to get agreement on.

  4. Get detailed specs for each task and work to them, don't add unspecified 'nice to haves'. These can be separate tasks. If you don't have a detailed spec, ask questions. What colour should the button be? font, sample test data for a function, should this edgecase be included? etc.

  5. Ask how long the last similar project/task took and use that as your estimate.

Scrum can make development super easy and laid back if you have a well thought out plan and stick to doing the tasks as specified. But gets crazy when you get too caught up in the processes

5
  • 1
    The whole idea of task estimation is to estimate how expensive a task is. A dead line MUST NOT ever play any role here, since a task doesn't get any less work, just because the deadline is close. If a task is a lot of work, it always will be and then your estimate cannot be anything but that. And managing deadlines is not your task as a developer and thus not your problem. If the task doesn't fit into the deadline, the deadline must be shifted or the feature will be missing from the shipped product, those are the options for product owner and which one is chosen is not your problem.
    – Mecki
    Nov 22, 2023 at 16:31
  • 1
    I think you are being overly optimistic. If OP is being questioned about his estimates, then the manager obviously has a view as to what they "should" be. By asking about the managers deadline, you can come to an understanding with them and trim features or adjust expectations if needed. It's a way to destress an argument about estimation.
    – Ewan
    Nov 22, 2023 at 16:40
  • If the manager thinks he knows better how much work is to implement a feature, then he's free to implement it himself. Considering that most managers can barely program Hello World, I wonder how they think they know better the amount of work requires than those people they hired as experts to perform that work. Let me guess, they also tell their doctors how the treatment should be performed and their car mechanics how to fix their cars, despite having zero knowledge about medicine or car mechanics?
    – Mecki
    Nov 22, 2023 at 17:22
  • yeah, the idea is not to fight with the PM about how scrum works. Thats what causes stress, try to understand why they are upset and work out a solution together as a team.... like on bob the builder
    – Ewan
    Nov 22, 2023 at 20:27
  • An idea of a deadline by management has no impact on my estimates. They are my estimates based on my understanding of requirements and complexity. Its those or you give on requirements. Very few tasks ever have "detailed" requirements. User should be able to do this is usually where that ends without significant effort is not detailed. Bulk estimate in my experience reduces accuracy. Management is paying me for my expertise and part of that is in understanding what it takes to accomplish X and Y tasks. If they know better they probably are better engineers than me. Rare they are engineers.
    – Rig
    Nov 30, 2023 at 19:09
6

Scrum feels very different depending on how you do it. In my experience, nobody does "perfect" Scrum in accordance with the guidelines in every aspect. And that is a good thing. It is a framework, not a religion.

I have had very good and mediocre experiences with Scrum, but I also heard a lot of negative experiences from co-workers.

The practices you describe are very toxic and unhelpful ways to employ "Scrum". This and very similar situations often result from either top management or customers telling the manager to go "agile" and "use Scrum" while they just LOOOOOVE their waterfall and did nothing else in their career.

The most important thing when doing Scrum aren't the Rituals/Meetings or something like that. It is an agile mindset of EVERYONE involved: Devs, Product owner, manager. Without that (or the collective intention to get to such a mindset) Scrum and any other agile method is doomed to fail and result in waterfall with a nice coat.

Now to answer the question

Estimates:
In every good Scrum experience (without exception), we didn't estimate in time, but rather in abstract "Story Points" (short SP) using the fibonacci numbers. Even though, in the end, these SP somehow correlate to dev time. Using this abstraction layer keeps the expectations about the estimations more agile.

As you said, you always argue about these estimations and doing them in hours guarantees that no real consensus can be found. The senior would be able to do the ticket in 4 hours while the fresh junior will take two days. Both are probably right with their estimation, IF they are the ones doing the ticket.

Sizing the ticket in SP makes clear that you are talking about how big the ticket is in relation to other tickets and it is much easier to get a consensus that the ticket is 3 SP big. The senior dev just does more SP per time than a junior. Also two tickets rated at the same size can still take different amounts of time, it is just an estimation after all.

Burndown:
Burndowns are screaming "waterfall" at the top of their lungs. One of the goals of Scrum is to replace burndown-chart management. You plan a sprint and the only thing that matters is that at the end of the sprint the sprint goal is met.

If not, the retrospective is the time to talk about why you didn't finish all of the tickets and how this can be prevented in the future. Don't play the blame game. Have a constructive discussion to find the reasons, and always start with the assumption that everyone gave their best.

If burndown charts are "neccessary" then it is a sign that your sprint cycle might be too long. Talk with your team about trying a shorter rhythm.

For the manager to keep an eye on the progress, it is much better to simply attend the dailies and retrospectives.

2
  • 1
    "Even though in the end these SP somehow correlate to dev time, using this abstraction layer keeps the expectations about the estimations more agile." – What I like about thinking about SP in terms of complexity rather than time, is that more complexity automatically also means higher risk. Therefore, when you think about SP in terms of complexity, it should be obvious that higher SP not only means more complexity and, yes, ultimately in some way, shape, or form, also longer time, but it also means more uncertainty about the estimate in the first place. Nov 22, 2023 at 16:26
  • I don't know that I completely agree on burn down. A burn down chart can be an indicator of a change in tempo or a sign of a broken process. If the board is maintained well cliffs and mountains can reveal things like a massive task dump at the end of a sprint for QA that isn't being accounted for or sandbagged estimates. All kinds of things. You shouldn't run your process based on it but it can provide hints on process improvement. Maybe the tickets aren't broken down correctly or the requirements have been bad. Host of things it could hint at though maybe not be conclusive evidence.
    – Rig
    Nov 30, 2023 at 19:14
5

This is entirely from some good and some bad experiences in a formal Scrum environment.

Do I have to accept that Scrum is annoying/stressful based on the above description, or is there a way to deal with it differently which might lead to a different experience?

I can offer a few points to consider:

  • Are all of the other members really making their estimated deadlines? Bad as it may be, misery loves company and having open discussion with other developers in your group about this may get rid of some of the stress.
  • Feeling pressure to estimate poorly is common but also stressful. You may do better simply sticking to your, "real," estimate. Sometimes it's not bad to let someone else do a backlog item and take something easier yourself.
  • In the good Scrum environment that I worked in (yes, singular, in over 25 years of this), the team was friendly in a competitive way rather than cutthroat. Stand-ups were fun. We all did a little to have at least superficial personal relationships (happy hour, events outside work, etc.) so we saw each other as people rather than enemy entities when things got bad. They can be fun but this is difficult and takes work on everyone's part.
  • I don't know where you are but here, we bring donuts. I've told people in the past, "I brought these so that you'll like me more and the meeting will go easier," and it's always worked. Keep it ethical but gifts really do work.

How can I overcome the bad conscience when I’m wrong with an estimate or when I don’t manage to finish the planned tasks by the end of the sprint?

Mostly, you can't overcome feeling badly when you make a mistake. It's part of the human condition. Some of these things may help you see things differently:

  • Remember that missing deadlines is only a personal issue if you don't work to correct the way you do things. I consider most mistakes as, "first one free," as long as there were good intentions. Most of us learn best from mistakes.
  • Stick to your real estimates. Politicians tell people what they want to hear. Software development engineers tell people how bad it really is. Be the SDE instead of the politician. There are organizations where this is valued.
  • Ultimately, you have to be able to disassociate at least a little. I don't know where you are in your career but this has been easier and harder for me at different times in mine. No one has a perfect track record and it takes work to learn and accept that.
  • I do not know you but very often the determination that goes into making sure the job is done well comes with hesitancy to admit one's shortcomings. (This is a lifetime struggle for me and I have met many others like this.) Can you think of more ways in which your manager is right when they confront you about missed estimates? Would it be possible to learn more from them or others? Many very successful people in most fields have mentors throughout their careers, which can help in the perspective that we always have more to learn.

Also, some notes that may help to reduce the number of mistakes in your estimates:

  • Try to pick some, many or all backlog items that are routine. Smaller issues that seem more trivial are easier to estimate. Finish them quickly and without gold plating. It will help your outlook, how you are perceived, your numbers, and the amount of time you have for bigger items.
  • Try to avoid, "R&D," projects. They're the sort of thing where you don't know the answer before you start, or the problem hasn't been solved before. They're the most interesting projects for many of us but they're the ones most likely to blow out estimates.
  • If you must do R&D projects, see if your leader will reclassify them. Sometimes they have an incorrect granularity (does adding a new reporting scheme really fit in one sprint?). If you can't get them reclassified, and you know in advance, call out the big, complicated, unknown parts of an item. There are better solutions than accepting a silent death slog into a black hole.
  • Add padding to your estimates. This takes a little discipline to not pre-underestimate because you know the padding is coming. It's a cheap trick but it has worked for many people I've met.
  • S/M/L estimates, suggested in some other answers, can be extremely effective. They also save time.
  • Practice and experience are the best solution. If you can, estimate other items that aren't yours, then compare your estimate to what came up at your stand-up. Possibly, ask one or more other team members if they would talk you through their estimating process. Doing estimating more is unlikely to make you worse at it.
0
2

Simply, the whole purpose of these practices is to put pressure on developers and sweat them.

It's supposed to exhaust you, because in a certain school of management thought, that is the ultimate proof that they are extracting the maximum possible value from you.

The assumption is that useful work is tiring, and that workers don't like to render work, at least not to the full extent they can.

The fact that there are overheads, like draining haggling about "estimation" (which are treated more like deadlines), and the fact that behaviour alters in that quality is neglected or things that require thoughtful consideration get rushed or skipped, are largely just ignored or assumed to be insignificant problems.

The fundamental assumptions that drive this mentality in management are often prevalent in a particular company, and rarely up for serious re-evaluation.

The best way to deal with it is to move to a different company.

7
  • 4
    The tone of this answer is disproportionately cynical. I'm not discounting it, because this kind of thing definitely happens in the real world, but it's an overstatement to say that scrum is inherently designed to unreasonably pressure developers. It's not. A manager who is so inclined to put more pressure on their developers can do so using any tool at their disposal, regardless of it being scrum or something else. Overall, this answer seems to succumb to "my specific experience is the only possible experience", making it a bad answer that lacks nuance.
    – Flater
    Nov 22, 2023 at 2:27
  • That being said, I do actually support the notion that moving company might be warranted if management has taking this kind of attitude towards its treatment of developers, but not simply because the company uses scrum. Scrum is being unfairly tied to an already toxic environment in this answer.
    – Flater
    Nov 22, 2023 at 2:30
  • @Flater, I haven't actually worked under these conditions, and I'm not attempting to talk about "Scrum" in the abstract (a word you'll see I didn't actually use in my answer), but in the form of the concrete practices the OP describes, which he says are called Scrum where he works (and which I know are often called that elsewhere).
    – Steve
    Nov 22, 2023 at 9:18
  • 2
    "the whole purpose of these practices is to [..]" and "I'm not attempting to talk about [..] in the abstract" are contradictory. You left no room for nuance by using such a blanket statement. I'm also discounting the whole "I didn't say scrum" cop out. In a question about scrum, referring to "these practices" without any further qualifier will be understood to refer to the topic of the question, not some arbitrary implicit similar-but-different topic that appears only when your initial claim is being countered and is used to evade said counter as if it were non-applicable.
    – Flater
    Nov 22, 2023 at 11:34
  • 1
    "Who governs?" is the real question, or as Lewis Carroll wrote in Through the Looking Glass "The question is which is to be master - that's all". Most battles over "method" are really about the extent to which non-technical management dominate over the development work versus the practitioners who actually do the work. The owners of financial capital and their management emissaries, versus a powerful and expensive craft workforce who are difficult to treat as interchangeable and whose ongoing involvement is somehow necessary for the good health and profitability of a business operation. (2/2)
    – Steve
    Nov 22, 2023 at 12:17
2

Your experience with Scrum, as you've described it, resonates with a common dilemma in the software development world. It's clear that the principles of Scrum, which are intended to streamline and enhance productivity, are instead contributing to your stress and diminishing your job satisfaction. Here's my take on your situation and some advice moving forward.

Addressing Scrum's Stress Factor

Scrum, in its essence, is a framework designed to be flexible, iterative, and adaptive. It's not inherently stressful, but the way it's implemented can make a significant difference. The issues you've highlighted, such as the pressure of estimates, the burndown chart's relentless tracking, and the cycle of retrospectives, are often symptoms of a deeper problem: the workplace culture and the management's approach to Scrum.

1. Changing Perspective on Estimates and Deadlines:

Estimates are indeed just that—estimates. They are best-guess scenarios meant to guide, not dictate. It's crucial to foster a culture where estimates are understood as flexible, and revisions are accepted as part of the process. This understanding can alleviate the pressure and guilt associated with 'incorrect' estimates.

2. The Burndown Chart: A Tool, Not a Judge:

The burndown chart is a tool for visualization, not a mechanism for penalizing team members. It should be used to identify bottlenecks and facilitate problem-solving, not to induce guilt or stress.

3. Retrospectives: Learning, Not Blaming:

Retrospectives are meant to be learning experiences, where the team collectively reflects on what worked and what didn’t, without pointing fingers. It’s a time for constructive feedback and collaborative improvement, not justifications or blame.

Moving Forward: Reevaluating Your Workplace

Given your experiences, it may be worth reconsidering if your current work environment aligns with your professional values and work style. Scrum, when practiced in a supportive, understanding environment, shouldn't drain your energy or enthusiasm. If the implementation of Scrum in your current company is rigid and counterproductive, it might be time to explore opportunities where the work culture and methodologies align better with your preferences.

Remember, no job is worth sacrificing your well-being and professional satisfaction. Sometimes, the best course of action is to find an environment that respects and values your contributions and understands the true spirit of agile methodologies like Scrum.

I hope this perspective helps you in making an informed decision about your future career path.

0

To paraphrase your questions:

  1. Is Scrum annoying/stressful?

  2. How to overcome bad estimates

Short answer - accept that Scrum has evident flaws and for a lot of organisations, it can be better to move on to something that actually delivers value in a far more efficient way rather than agonizing over how best to fill an arbitrary 2 week window.

But to answer the questions directly:

  1. Scrum can definitely feel annoying and stressful to some people. The main gripe is that you're asked to guess when something will land when there are a lot of unknowns. The answer here is to either get thicker skin or to limit the amount of work in a sprint. The latter is obviously easier than the former. SMs/PMs for their part need to understand this and stop using corrosive terms such as "failed sprints" and stop cramming sprints to capacity and making "hero moves" of bringing work forward.

  2. Forget the idea that there are bad estimates. Estimates were good at the time they were given. In reality they should be used as a comparative measure to fill sprints. However, they're regularly abused by many players who convert them to absolute time. The harsh reality though is that project X is usually required by date Y. If you're regularly missing deadlines, look at how estimates are done again or see why they slipped in the retrospective. I keep a personal log of items in the retrospective. If similar themes crop up time and time again, I escalate these as it is a clear sign that issues aren't being dealt with.


As a final aside, I've always thought a "sprint" was a terrible term. You're supposed to be able to maintain development pace forever. A sprint is the exact opposite of that.

-1

This is the type of question to get emotional and throw out all the frustration about a subject applied to Scrum.

Scrum is a methodology that became hype in software development, although it is applicable to any activity. Scrum along with its particularities has common ground with other software development methodologies, one is it makes an activity traceable, another one sometimes, I am unable to explain why, the unspoken purpose of using it is to boost moral and increase team’s cohesion, worthless to say such a passive-aggressive approach has the opposite effect on team’s moral.

How to make Scrum less stressful

TL;DR Add transparency about what developing software following a methodology implies.

Before start using a software development methodology it helps to state that software development methodologies are not about software development, they are a different activity impeding software development since part of the time for software development has to be invested in a different activity. To release the stress start from the thought that what has to be done is half something else than software development and half software development. Think of carrot and stick metaphor, it is same with do some physical effort to get endorfine (the reward).

very difficult to give estimates

Software development methodologies are not about people. With the knowledge at hand an estimate is expressed; different levels of knowledge result in different estimates; there is nothing to negociate. To release the stress acknowledge the estimate activity is based on the knowledge at the time the estimate is made and is not about the person estimating; estimate based on own knowledge not based on others’ knowledge nor on what it could have been known.

This means that you can immediately see "mercilessly" whether you are on track or behind schedule compared to the estimated time

Software development methodologies are not about people. The development progress is different than foreseen, there is nothing to justify. What might feel uncomfortable is to reiterate in retrospective all the effort the team went through during the reviewed sprint since everybody knows it talking about it is not going to change the result and that’s where disconfort springs from.

Do I have to accept that Scrum is annoying/stressful based on the above description, or is there a way to deal with it differently which might lead to a different experience?

When the sole interest is software development there is no way to feel comfortable about activities distracting attention from what is of interest. To release the stress acknowledge that part of the time has to be invested in something different completely unrelated with software development.

Summed up

How to make Scrum less stressful

Either don’t use it or agree to shift attention and invest most of the time in different activity than developing software. Pardon the gloominess.

Not the answer you're looking for? Browse other questions tagged or ask your own question.