When is it okay to ship a product with a known bug?
I assume you are talking of a "known" bug (the question is meaningless otherwise). Well, the answer depends on these factors:
1) Who is the user and how will he/they accept the bug in case it is found?
2) What is the potential impact (damage) of the bug?
3) Is it feasible to delay shipment of the software in order to fix the bug?
4) Most importantly: what do you expect from your software? Time-to-market? Quality? Do you want to see if your customers use the software long enough to find the bug?
Its a judgement call. Remember, a bug can be many things. If its a major piece of functionality thats flat out not working, then you fix it first. If its something minor that has minimal or no real impact on the program's usefulnes, you might let it slide.
So look at it from a cost/benefit viewpoint.
You ship products with known bugs when the total cost and risk of fixing the bug outweighs whatever problems and negative impact would arise from the bug being out there.
So if you have a 2 week testing period before you release, and a small bug is found, the question is... is fixing that bug worth the 2 additional weeks a team might now have to spend to re-test the application and installation (an often forgotten about step in creating software)? What are the cost to reputation or in sales if the software is late? Are people going to be angry? They might be quite happy to live with a minor bug if the major functionality can be delivered on time.
Risks include the risk of introducing a new problem, not just in fixing the bug, but also things that might arise from creating a new installation.
Negative impact is both the time and effort dealing with customers reporting the bug, and things like reputation damage.
Bugs come in different severities. At any software companies I've worked at we've categorized bugs in order of Priority from P0 to P4.
P0 Is the software does not work/crashes and could cause damage to the customers business. P1 It's not working as designed and crashes consistently in core functionality P2 It crashes occasionally and or a piece of side functionality may not work. P3 Some element of the software is not working as designed/expected P4 Cosmetic issue.
I've worked in places where P4's just don't get fixed because they have such a small effect on the software.
I'd say it's okay to ship if your software has known P3/P4 issues, I'd put this in the release notes and note that they are being worked on.
I'd never release software with a P0,P1 or P2 issue that I was aware of.
It's called a "Known Issue". Google, Microsoft, Apple, etc all ship products with bugs, both known and unknown. Try to minimize them, but don't wait for perfection. Ship fast, ship often.
when it's a "feature"! ;)
On a serious note, unless you are a perfect programmer with a perfect test setup, to perfectly test every single code path and eventually that could potentially exist, it's improbable that you will ship a bugless product.
Therefore, be pragmatic, if everything you've encountered in your testing has been fixed, anything extra should be fixed on an as needed basis.
It is frequently better to have a product ship on time, with a list of known issues, than not ship at all.
One of the things in the programming world that gives people confidence in a project is whether they have release scheduled and whether the schedule holds.
This is why Ubuntu ships releases every half year, even if there are still open issues.
I'd say that a good rule of thumb is, "is this bug a showstopper?"
If the bug causes a "happy-path scenario" to fail, then absolutely do not ship with that bug.
If the bug causes a "tangent-to-the-happy-path scenario" to fail and there is no workaround, then do not ship with that bug.
If a bug is documented and there is a known workaround, then it is probably OK to ship with that bug.
From a consumers point of view... Never. My point is as long as you know there is a major bug in the software then you should never ship it. However forces of nature (business) override this if the production cycle of the software is now at the stage where it can harm the business model and they are minor bugs that wont: (i) compromise the security of the software (ii) impact usability
As people have said, its a very broad question. It actually takes me to an interesting perspective: the so called "bugs" you claim is only the faults that were discovered by your testers. There could be an infinite more loopholes. Just reminded of an interesting story I heard from a respected Professor in one Graduate Seminar: When people in one of the Scandinavian countries used a "handwriting -recognizable" type voting machine, certain people hacked the entire system by writing malicious SQL code (which the system took in as normal input).
There's something called the FMEA (Failure mode and effects analysis) it's very useful to decide when a known bug is important or not based on:
Another deciding factor can be how the defect relates to your last release. If the bug is part of a new, but broken feature, then the non-functionality doesn't represent a regression of functionality. Go ahead and ship.
On the other hand, if the defect causes a loss existing functionality that is known to be of use to existing customers, then it must block the release. Such a release would be a downgrade for your customers, and would serve neither your interests nor theirs.
There can be shades of grey in this. A regression in core functionalty should never ever go out the door. Some regression in peripheral features could go out to lead users if the release also contains new functionality they have expressed interest in. An obscure defect that is not likely to affect many customers can go into a release, so long as a work around is provided when it does affect those customers. Defects in highly experimental features that are turned off by default anyway should never cause the release to be delayed.