20

When is it okay to ship a product with a known bug?

Improve this question
12
  • 5
    The question is probably too broad. The reasons why are infinite.
    – jmq
    Mar 11, 2011 at 8:14
  • 7
    The question is: ship with bugs or not ship at all :)
    – Vitor Py
    Mar 11, 2011 at 11:12
  • 41
    All products ship with bugs.
    – Joel Etherton
    Mar 11, 2011 at 11:28
  • 4
    Define BUG. We recently shipped a product that wouldnt install. Great bug :D
    – Barfieldmv
    Mar 11, 2011 at 13:31
  • 2
    Do you mean 'known bugs'?
    – Nobody
    Mar 11, 2011 at 15:29

14 Answers 14

Reset to default
12

I assume you are talking of a "known" bug (the question is meaningless otherwise). Well, the answer depends on these factors:

1) Who is the user and how will he/they accept the bug in case it is found?

2) What is the potential impact (damage) of the bug?

3) Is it feasible to delay shipment of the software in order to fix the bug?

4) Most importantly: what do you expect from your software? Time-to-market? Quality? Do you want to see if your customers use the software long enough to find the bug?

Improve this answer
119

It has to always be OK, because there is no such thing as bugless software.

Improve this answer
14
  • 2
    but it sounds like he's aware of the bug... so at that point it would seem that a programmer is just being lazy to not fix it being aware of it...
    – Kenneth
    Mar 11, 2011 at 8:16
  • 7
    @Kenneth: You could see it like that, but products have to be shipped, and they will always have bugs. It would depend on the severity of the bug as to whether you hold off on your deadline.
    – Matt Ellen
    Mar 11, 2011 at 8:19
  • 1
    @Matt this is true. However it would seem to me that in most cases you wouldn't want to knowingly ship a product with major bugs. That would mean that the remaining bugs that you know about would be minor which in most cases would be easily fixed or at least worked around. You can't handle bugs you don't know about but if the testing process is being done correctly most bugs should be caught...
    – Kenneth
    Mar 11, 2011 at 8:26
  • 1
    Maybe my sarcasm wasn't clear... but saying that it's "always OK" to ship buggy software is kind of irresponsible. It's like saying "there will always be murderers in the world, so it doesn't matter if I go and kill a few people".
    – DisgruntledGoat
    Mar 11, 2011 at 13:17
  • 7
    @DisgruntledGoat Not all bugs are equal: some are easy fixes, some are project destroying disasters. Obviously those should be fixed. Some are very rare bugs, hard to find, based on unusual circumstances, and usually difficult to fix without major rewriting. Sometimes those just have to stay in because fixing them offers too little value and the software needs to ship yesterday. Its all about cost/risk/benefit analysis.
    – CodexArcanum
    Mar 11, 2011 at 14:54
33

Its a judgement call. Remember, a bug can be many things. If its a major piece of functionality thats flat out not working, then you fix it first. If its something minor that has minimal or no real impact on the program's usefulnes, you might let it slide.

So look at it from a cost/benefit viewpoint.

You ship products with known bugs when the total cost and risk of fixing the bug outweighs whatever problems and negative impact would arise from the bug being out there.

So if you have a 2 week testing period before you release, and a small bug is found, the question is... is fixing that bug worth the 2 additional weeks a team might now have to spend to re-test the application and installation (an often forgotten about step in creating software)? What are the cost to reputation or in sales if the software is late? Are people going to be angry? They might be quite happy to live with a minor bug if the major functionality can be delivered on time.

Risks include the risk of introducing a new problem, not just in fixing the bug, but also things that might arise from creating a new installation.

Negative impact is both the time and effort dealing with customers reporting the bug, and things like reputation damage.

Improve this answer
9
  • 30
    A typo in the 'about' box is something you really should fix. It will take about 0.7 seconds (assuming we both type at the same speed). However if you leave that typo in, people can see it. It's instant death for your users' confidence in your software if there are visible mistakes in the user interface.
    – user4051
    Mar 11, 2011 at 9:17
  • This seems about right to me. Most of the time there are a few known minor bugs in a product even when it goes to release. They tend to be things that are very unusual and have a negligable effect on the actual operation and use of the software or things that most users never see.
    – glenatron
    Mar 11, 2011 at 10:40
  • 3
    Indeed, typos in your UI shatter faith in the quality of a product (wrongly, as many great programmers just don't speak good english), however I see your point - trivial bugs that cause no real harm and probably won't ever come up aren't worth the hassle of holding back a release. Leave it for a bullet point in 1.01.
    – Phoshi
    Mar 11, 2011 at 10:58
  • Do not let spelling errors into your application. Quite frankly I'm more embarrassed by them then an actual bug.
    – ChaosPandion
    Mar 11, 2011 at 19:04
  • 1
    I have no idea what you are talking aboot... ;)
    – Andreas Johansson
    Mar 11, 2011 at 19:14
6

Bugs come in different severities. At any software companies I've worked at we've categorized bugs in order of Priority from P0 to P4.

P0 Is the software does not work/crashes and could cause damage to the customers business. P1 It's not working as designed and crashes consistently in core functionality P2 It crashes occasionally and or a piece of side functionality may not work. P3 Some element of the software is not working as designed/expected P4 Cosmetic issue.

I've worked in places where P4's just don't get fixed because they have such a small effect on the software.

I'd say it's okay to ship if your software has known P3/P4 issues, I'd put this in the release notes and note that they are being worked on.

I'd never release software with a P0,P1 or P2 issue that I was aware of.

Improve this answer
5

It's called a "Known Issue". Google, Microsoft, Apple, etc all ship products with bugs, both known and unknown. Try to minimize them, but don't wait for perfection. Ship fast, ship often.

Improve this answer
3

You can't ship software without bugs. The advice I can give is that it always better to say to your customer: "This version can't do that and that but we are going to fix this" than the situation when customer tells YOU that you have a bug.

Improve this answer
0

when it's a "feature"! ;)

On a serious note, unless you are a perfect programmer with a perfect test setup, to perfectly test every single code path and eventually that could potentially exist, it's improbable that you will ship a bugless product.

Therefore, be pragmatic, if everything you've encountered in your testing has been fixed, anything extra should be fixed on an as needed basis.

Improve this answer
0

As long as you are honest to your customers you can ship with bugs. Telling them about all existing bugs shows them that you have good knowledge about your software, and that is indeed is well tested (if it is..). :)

Obviously the best thing is to avoid shipping with bugs.

Improve this answer
0

It is frequently better to have a product ship on time, with a list of known issues, than not ship at all.

One of the things in the programming world that gives people confidence in a project is whether they have release scheduled and whether the schedule holds.

This is why Ubuntu ships releases every half year, even if there are still open issues.

Improve this answer
0

I'd say that a good rule of thumb is, "is this bug a showstopper?"

If the bug causes a "happy-path scenario" to fail, then absolutely do not ship with that bug.

If the bug causes a "tangent-to-the-happy-path scenario" to fail and there is no workaround, then do not ship with that bug.

If a bug is documented and there is a known workaround, then it is probably OK to ship with that bug.

Improve this answer
0

From a consumers point of view... Never. My point is as long as you know there is a major bug in the software then you should never ship it. However forces of nature (business) override this if the production cycle of the software is now at the stage where it can harm the business model and they are minor bugs that wont: (i) compromise the security of the software (ii) impact usability

Improve this answer
0

As people have said, its a very broad question. It actually takes me to an interesting perspective: the so called "bugs" you claim is only the faults that were discovered by your testers. There could be an infinite more loopholes. Just reminded of an interesting story I heard from a respected Professor in one Graduate Seminar: When people in one of the Scandinavian countries used a "handwriting -recognizable" type voting machine, certain people hacked the entire system by writing malicious SQL code (which the system took in as normal input).

Improve this answer
0

There's something called the FMEA (Failure mode and effects analysis) it's very useful to decide when a known bug is important or not based on:

  1. Occurrence
  2. Severity
  3. Detection
Improve this answer
0

Another deciding factor can be how the defect relates to your last release. If the bug is part of a new, but broken feature, then the non-functionality doesn't represent a regression of functionality. Go ahead and ship.

On the other hand, if the defect causes a loss existing functionality that is known to be of use to existing customers, then it must block the release. Such a release would be a downgrade for your customers, and would serve neither your interests nor theirs.

There can be shades of grey in this. A regression in core functionalty should never ever go out the door. Some regression in peripheral features could go out to lead users if the release also contains new functionality they have expressed interest in. An obscure defect that is not likely to affect many customers can go into a release, so long as a work around is provided when it does affect those customers. Defects in highly experimental features that are turned off by default anyway should never cause the release to be delayed.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.