183

My first commit in my project resulted in the nightly build being broken and people are all over me as we are nearing the release. I want to send an apology email that should sound sincere and at the same time hinting that this was my first commit and this would not be repeated any more.

Being a non-native English speaker, I have difficulties coming up with correct words. Can someone please help?

24
  • 103
    If the build never broke I'd begin to suspect a broken build process ;)
    – Lazarus
    May 25, 2011 at 11:54
  • 7
    How were you to know that the build was broken?
    – user1249
    May 25, 2011 at 12:01
  • 68
    How about : "I am very sorry for breaking the night build. If you want to withold my salary as a form of punishment, I will not take the company to employment tribunal". Nah, just kidding - do not apologize, this sort of stuff happens all the time. People who are "all over you" are f.king psychos who don't know how to properly restore the system to previous state.
    – Jas
    May 25, 2011 at 12:44
  • 14
    I broke the build on my first 10 commits! Dont worry about it
    – Nobody
    May 25, 2011 at 13:02
  • 137
    I just wish i had a nightly build to break..
    – Max
    May 25, 2011 at 15:53

23 Answers 23

307

Don't apologize!

  • Breaking the build once in a blue moon is not a big deal, and should never be a show-stopper.
  • It's your manager's fault for not configuring continuous, automated builds.

Also, I bet your team fails the 'Joel Test' and can't make a build in one step.

If so, this would be another thing that you shouldn't apologize for. Indeed, it's a team anti-pattern.

12
  • 32
    +1 Agreed! Don't apologize. LEARN what you did wrong. Don't do it again, at least not soon. Be polite when people are "all over you". Learn from what they say (even if it's just who's a prick and who's OK).
    – Peter K.
    May 25, 2011 at 11:53
  • 13
    Well, you can apologize still... But I'm a bit surprised that people are all over you. If you are new on the team, it shouldn't be such a surprise that you made a mistake. At least it shows that you did something :)
    – Philippe
    May 25, 2011 at 12:17
  • 43
    +1. The whole purpose of doing a nightly build is to catch problems as early as possible and before they go sailing out with a release. 95% of developers have made high-visibility mistakes during their careers. The other 5% are lying.
    – Blrfl
    May 25, 2011 at 14:52
  • 26
    While I agree that this does not call for a "falling on the sword" level apology, I think it does call for an "oops, my bad" level apology. Not every 'I'm sorry' needs to be abject. May 25, 2011 at 17:15
  • 5
    I don't agree with this premise at all, there is nothing wrong with a simple "I'm sorry" apology, I realize I screw up and will do my best to learn from my mistake. I agree the best way to * amend* yourself is not to do it again, but you if you care, you might wan't to openly show it to let others know, not every time you screw up, but he clearly felt bad about it and there is nothing wrong with apologizing.
    – Trufa
    May 26, 2011 at 14:37
184

Bagels. Donuts. Etc. At one company I worked for in the past, checking in broken code or otherwise causing colleague disruption is generally resolved by the bringing in of apology foodstuffs the next day.

We had a guy blow away a production database one day, causing massive panic and a late night for the whole team. The next day he grilled burgers for lunch.

I love co-worker apologies. Tasty, tasty apologies.

15
  • 84
    +1 for "I love co-worker apologies. Tasty, tasty apologies."
    – Jas
    May 25, 2011 at 12:40
  • 33
    Breaking the nightly dev build is one thing but blowing away the production database is on a whole other level. If I ever did that I wish that grilling burgers was the worst of my punishment.
    – maple_shaft
    May 25, 2011 at 12:42
  • 21
    You can almost taste the guilt.
    – MSpeed
    May 25, 2011 at 13:14
  • 42
    "Blowing away a production database" puts all kinds of hilarious images in my head as to what exactly happened to the database. Most of them involve everyone hearing a loud explosion from the server room. "Oh god, the database is reaching critical volume!" "Quick! Lower the control rods!" "It's too late, the data, she is doomed!" BOOM May 25, 2011 at 20:57
  • 9
    drop database ... Oh the power of those two little words. It was years ago, but I remember it well ;)
    – Leigh
    May 26, 2011 at 18:10
81

Two quotes for you:

The man who makes no mistakes does not usually make anything.--William Connor Magee

Anyone who doesn't make mistakes isn't trying hard enough.--Wess Roberts

I agree with Jim G., don't apologise but do learn from it and don't make the same mistake again... keep it DRY ;)

4
  • 9
    Or there is the more general quote of: "let he who is without sin cast the first stone" If who ever is moaning about the broken build has broken the build before, they shouldn't be moaning
    – Richard
    May 25, 2011 at 16:46
  • like the second one!!
    – Sufendy
    May 26, 2011 at 2:24
  • 2
    Quoting myself to every Grad/junior I've ever mentored, "I expect you to make lots of mistakes. Own up to them, accept them, and learn from them. If you never make mistakes, you'll never really learn anything".
    – S.Robins
    May 6, 2012 at 23:37
  • Nice use of the "DRY" principle... :)
    – J. Allan
    Oct 21, 2016 at 19:20
54

Don't apologize, just FIX IT as soon as possible. It is okay though, everybody breaks the build at least once, in my last company it was something of an initiation ritual. When a team member broke the build we would put a rubber duckie on his desk in the morning before he came in, this let him know he broke the build and he would fix it.

We called it the Continuous Integration Duckie and when you had it for the day people would tease you but it was all in fun, none of it was supposed to be mean spirited.

We took something like a broken build and turned it into a team building exercise.

3
  • 3
    +1: not just what they care about - all they should care about. You haven't done anything wrong, as such - just flexed the boundaries of what's possible a bit too far ;). You're probably also the person best placed to fix it as well. Everyone does this every now and again. Everyone uploads something to live that shouldn't have gone. Everyone leaves the WHERE clause out of an UPDATE statement once in their lives. It's how you react that's important. Where we are, all the devs tend to close up, refusing to speak up about who individually was at fault if anyone asks, it just gets fixed.
    – Tom Morgan
    May 26, 2011 at 8:17
  • That seems like a really great way to handle mistakes.
    – Trufa
    May 26, 2011 at 14:39
  • 3
    Continuous Integration Duckie, Hahahaha May 26, 2011 at 22:47
44

"Sorry! My bad!" is how I usually apologize when I've broken the build. It happens. But as others have said, this is an opportunity to improve your systems so that one person cannot so easily break the build for everyone else.

I would not make a formal apology in these circumstances, but if you actually feel that a more formal apology is appropriate, then your apology should do these things:

  • Express regret.
  • State the problem.
  • Take responsibility.
  • Make amends.
  • Save face.

That is, "I'm sorry [EXPRESS REGRET] that I inconvenienced you [TAKE RESPONSIBILITY] by accidentally [SAVE FACE] breaking the build [STATE THE PROBLEM]. Doughnuts are on me tomorrow. [MAKE AMENDS]"

Each part is necessary in a proper apology; if you don't state the problem then it is unclear. If you don't express regret, take responsibility, and make amends, then people feel like you're insincere. The face-saving part is the most overlooked part of an apology; the face-saving part is what reminds the injured party that you are a valuable coworker who sometimes makes mistakes, and not an idiot (or a saboteur!)

Finally, some thoughts on build breaking:

I work on the C#/Visual Basic compiler team. Of course today Visual Studio is now such a massive project that it has a team of its own just to manage build infrastructure, and a huge room with its own dedicated air conditioning system. Back in the mid 1990s when I started as an intern the Visual Basic build team was one intern -- me -- and a closet full of machines. Times have changed!

Back in those days before continuous integration and strong checkin processes, teams would have a variety of penalties for breaking the build. On some teams the penalty was that if you broke the build, you had to wear a funny hat every day at work until someone else broke the build. On some teams, if you were wearing that hat you were responsible for verifying that the nightly build was correct.

That last bit sounds maybe cruel, but it actually served a valuable purpose. Since almost everyone broke the build at one time or another, eventually the entire team would learn the processes for verifying the nightly build.

16

Don't apologize. It's your coworkers who are to blame for not reviewing your first commit and not having a quick feedback system for builds like a continuous integration server.

At my current job we have an informal rule that someone who sneakily commits just before leaving work and turns out to break the build has to bring candies/cakes/drinks for the whole team the next day. But we do have continuous integration that warns us of a broken build during the day. And the rule probably wouldn't apply to someone's first commit.

Anyway, a formal mail of apology is probably a bit too much.

1
  • Excellent point - peer review should have caught this. Because you do peer review changes before they're applied to master/HEAD/default, right? May 27, 2011 at 20:11
11

A fundamental rule - when you're wrong, ADMIT IT :-| You don't have to grovel in apologies. Everyone makes mistakes. The pros admit it. That's teamwork. The other team members should be pulling together to help you over it. If they don't, ASK for help. The most that has to be said afterward is - what can we learn from it?

They say a successful marriage is based on three little words - "I was wrong".

If someone doesn't make an occasional mistake, they're not working, but a mistake one doesn't learn from is two mistakes.

1
  • I think this is a great answer and much better than the "Don't apologise" one that has the highest votes. You can apologise to everyone for breaking the build, but they also need to show you a bit of grace. They should also be saying "don't worry, we've all broken it before, after all, that's what it's there for". As long as you try not to break it again, they should be cool with that. If you ignore everyone and go and do the same mistake then that's a different story.
    – Rocklan
    May 7, 2012 at 2:05
6

The best apology is fixing the break quickly

5

If your company already has a way to test your build changes, then (A) your changes either failed (but you checked them in anyway) or (B) they succeeded (and you need to build a new test case).

If your colleagues gingerly test their changes and expect to find breaks on the nightly build, then (C) you have a brittle process (and you need to introduce testing like that found in Extreme Programming).

It is possible that (D) Your changes caused unforeseen changes in Bill's code that were either in place before or changed on the same build as yours.

Depending on how you and your company test, I would apologize on a case-by-case basis:

  • (A) I failed the build test but checked in my changes. Sorry, I am changing my process so I won't do it again.
  • (B) I passed the build test and added JUnit test XYZtest.java to reduce the chance of re-occurrence.
  • (C) Since we don't have a build test process, I am creating one for my changes. I'd like to share with you how we can improve our build process.
  • (D) I will work with Bill to write a JUnit test XYZtest.java to reduce the chance of re-occurrence.

I'm sure there's an (E) that I haven't thought of.

Notice that I am saying "to reduce the chance of re-occurrence" rather than "so it won't happen again." It will happen again. But you may improve your process to reduce that possibility. That, I think, is one of the marks of a winning programmer.

2

Don't apologize. You are human and you will make mistakes. Everyone will break the build occasionally, the key is to just fix it quickly.

As for the people jumping all over you...well I'm curious as to if they've ever written in a bug.

2

How to approach this depends on the atmosphere in your group. If it's a blame culture, I'd be very careful about apologising and how you do it. If it's a collaborative, positive atmosphere, then yeah, something along the lines of "I messed up, I'm sorry. How can we avoid this in the future?" is probably a good idea.

In any case, a goof up like this should be accompanied by some sort of post-mortem to a) find out how it happened and b) how to minimise chances of it happening again.

I'm not familiar with your structure (I work in a very different environment) but ultimately, the reality is that occasionally, people make mistakes, and things get broken. You learn from the experience and move on.

2

In my environment if you broke the build you would get a good natured ribbing and some witty cometary. I'm not sure which English speaking country your in, but it could be that as a non native English speaker you are not getting the underlining nature of the comments.

Being from the other side as a Senior developer I once commented on a code review that some way of doing x "sucked" not because the code was bad but do to the structure of the project. It was more commentary to myself that I need to fix the structural problem. Only later did I discover that the Jr Dev though I was very angry at him due to my inaccurate flippant speech.

2

Um. You get the broken build token. Pass the rabies on to the next lucky recipient. Happens all the time. Quality is important, but mistakes are inevitable. Fix them. Move on. Shame the next poor bloke.

1
  • 1
    I've seen this a lot. The other is you get to "look after" the nightly build until someone else breaks it. This doesn't mean fixing the it, but figuring out why and who should fix it. May 26, 2011 at 8:48
1

As a general rule, I would say:

If your checkin causes a compiler error you would have caught yourself had you done a "get latest" before checking in, a simple "whoops, my bad" is in order. (especially if you stroll in at 10 the next morning, while everyone is deciding which changeset to roll back to)

If your checkin causes general unexpected behavior, even runtime errors, I don't think it should be held against you. It comes with the territory. As long as everyone's "get latest" generally gets passed their compilers, people really should not be throwing a fit (with a couple exceptions like deleting a database, deleting the server copy of the project and all the changesets, or anything else that is so dumb that people have to assume malicious intent).

1

The TEAM failed.

Your company needs more code review on a dev doing a first time build.

I just don't see how a team allows this to go on without them doing a review and offering some assurances along the way that you are doing things correctly.

Being close to release time is not an excuse, but a better reason to double-check new code.

If your release cannot be easily undone, there are even bigger problems with this group.

0

I don't understand why people are all over you. If the system is setup well, they should be able to tease out your changes / fix them very quickly.

But again, if you are one of those guys who broke windows build, Then ... I can't help. (It is incredibly hard to do btw - before anyone questions MS build philosophies, but now and then someone does it - and the whole company QA stops for a day).

And yeah - don't apologize. Just have a chat with your manager and make him understand that you learnt from what you have done.

0

Builds break all the time. Bugs and mistakes are a fact of life, and that's why you should have a process that minimizes effects of bugs and mistakes.

If a build breaking is that big a deal, it means your process is broken.

You should be doing continuous builds, not nightly builds.

1
  • +1 for "if a build breaking is that big a deal, it means your process is broken." On the other hand, you still have to deal with the broken process, and that means doing what you can to avoid breaking the nightly build until the process is improved.
    – Caleb
    May 7, 2012 at 0:20
0

Better a late answer than never...

As many others have said, don't apologize for breaking the build. Simply admit that it was you and get on with the job. This stuff would happen whether you were there or not, and nobody deserves to be treated poorly because of it. People react badly under pressure, so if you are able to remain calm and simply get on with the job, you'll stand out when it matters. My advice for when people give you a hard time is to simply avoid being defensive, defuse the situation by letting people know you are either on to the problem or you are quick to seek advice if you find you've become stuck.

Personally, I see a broken build as an opportunity.

  • If the build breaks and you can prove it's your fault, then it's likely to show that the continuous integration and build processes are doing their jobs. This is a good thing, as it validates your processes. If the build never breaks, I'd worry that there was something wrong with it.
  • If you've broken the build in a fairly common manner, and it happens to break in this way often, then perhaps something is missing from the CI process. This is an opportunity to improve your processes so that the common failure scenarios can be better managed.
  • If you've gone beyond the call of duty and really messed up the build with an obscure problem, then you have an opportunity to learn about something new that might be important enough to trigger a warning to the rest of the team.

So what I'm saying is that breaking the build occasionally means that people are doing their jobs. Sure you might have made a mistake, but if you use it as an opportunity to learn, you are doing your job simply by learning to do it better next time.

-1

Tell them they need CI builds per check-in. That way you wouldn't have to wait until the nightly to know it's broken. YEA!!! Tell them it is the process that is wrong, nothing else. You just identified a gap in their system.

But yea, make sure to fix it. Not a big deal. It is not in production. Just a worthless nightly.

-2

A usual practice in my company is:

  • Shouting "it wasn't me!" (usually CVS / SVN proofs otherwise)
  • Wearing a t-shirt saying "my-userlogin ist Schuld!" (yes, 50% of our developer's own such a shirt)
  • Claiming that it works in the local sendbox and all tests passed just fine

My company also has a nice way of handling such an "incident":

-2
  • I would not apologise.

  • I would blame CI lead for allowing me commit broken build.

  • There should be a CI process in place for stopping developers from committing broken code.

  • If local build fails, then it should not be allowed to enter build server.

2
  • 1) Not all projects are set up to do continuous integration. It's nice to have, and can help with situations like the OP's, but it's far from a given. 2) It never hurts to apologize even when it's not really a big deal, even when there are tools that could have prevented the problem. 3) Blaming someone else for a problem that you caused, whether or not it's really your fault, is a lousy idea.
    – Caleb
    May 7, 2012 at 0:16
  • There are two problems here: 1 - broken code on local machine. 2 - broken code on a build server. First problem is very minor as all developers make mistakes. Changes can be undone and there will be no impact on the system or the department. Second problem is likely to have a much greater negative impact on the system and the department.
    – CodeART
    May 7, 2012 at 0:40
-2

While I think some kind of apology can be in place, please don't say: "I will make sure this doesn't happen again." Because it will. And if it does it will bite you.

-2

If you are working on an open source project,

just say "Sorry, I broke the build. May be I was too sleepy!"

and add it as a github comment.

It is because many open source developers write code during midnight.

Not the answer you're looking for? Browse other questions tagged or ask your own question.