Skip to main content
(Removed duplicate word.)
Source Link

Another consideration might be the type of test or testing organization that uncovers the bug, when trying to determine the impact and cost of the bug and fix. Unit or functional testing may point to something in the design that needs to be changed, and thus correcting early would be easier and less costly than later. System or integration testing may point to something that affects a wider variety of customers. And Standards testing, while often something non-critical to a large subset of customers, may lead to a loss of certification if not corrected and have a negative impact on the business.

That said, just because a particular test organization has a test fail shouldn't automatically make a bug "critical." For example, there might be a temptation to say something like "all system tests must pass before shipping, thus any system test that fails automatically results in a critical/high-severity bug." Hopefully no organization would make that statement (good test exit criteria is a different discussion), but the point is that the "severity" of bug should be judged on its impact to the customer, product, or company image rather than where or when it is uncovered.

One possible way to address that is to make a distinction between "severity" and "urgency." For example, as the release date nears there can be time pressure to determine if an apparently lower-severity bug might affect a large subset of customers, giving the bug a greater "urgency" and elevating work on that bug above (some) others at least until that determination can be made. A proper balance balance between the two, along with experience and good judgement, would help direct where time and effort is spent.

Another consideration might be the type of test or testing organization that uncovers the bug, when trying to determine the impact and cost of the bug and fix. Unit or functional testing may point to something in the design that needs to be changed, and thus correcting early would be easier and less costly than later. System or integration testing may point to something that affects a wider variety of customers. And Standards testing, while often something non-critical to a large subset of customers, may lead to a loss of certification if not corrected and have a negative impact on the business.

That said, just because a particular test organization has a test fail shouldn't automatically make a bug "critical." For example, there might be a temptation to say something like "all system tests must pass before shipping, thus any system test that fails automatically results in a critical/high-severity bug." Hopefully no organization would make that statement (good test exit criteria is a different discussion), but the point is that the "severity" of bug should be judged on its impact to the customer, product, or company image rather than where or when it is uncovered.

One possible way to address that is to make a distinction between "severity" and "urgency." For example, as the release date nears there can be time pressure to determine if an apparently lower-severity bug might affect a large subset of customers, giving the bug a greater "urgency" and elevating work on that bug above (some) others at least until that determination can be made. A proper balance balance between the two, along with experience and good judgement, would help direct where time and effort is spent.

Another consideration might be the type of test or testing organization that uncovers the bug, when trying to determine the impact and cost of the bug and fix. Unit or functional testing may point to something in the design that needs to be changed, and thus correcting early would be easier and less costly than later. System or integration testing may point to something that affects a wider variety of customers. And Standards testing, while often something non-critical to a large subset of customers, may lead to a loss of certification if not corrected and have a negative impact on the business.

That said, just because a particular test organization has a test fail shouldn't automatically make a bug "critical." For example, there might be a temptation to say something like "all system tests must pass before shipping, thus any system test that fails automatically results in a critical/high-severity bug." Hopefully no organization would make that statement (good test exit criteria is a different discussion), but the point is that the "severity" of bug should be judged on its impact to the customer, product, or company image rather than where or when it is uncovered.

One possible way to address that is to make a distinction between "severity" and "urgency." For example, as the release date nears there can be time pressure to determine if an apparently lower-severity bug might affect a large subset of customers, giving the bug a greater "urgency" and elevating work on that bug above (some) others at least until that determination can be made. A proper balance between the two, along with experience and good judgement, would help direct where time and effort is spent.

Source Link

Another consideration might be the type of test or testing organization that uncovers the bug, when trying to determine the impact and cost of the bug and fix. Unit or functional testing may point to something in the design that needs to be changed, and thus correcting early would be easier and less costly than later. System or integration testing may point to something that affects a wider variety of customers. And Standards testing, while often something non-critical to a large subset of customers, may lead to a loss of certification if not corrected and have a negative impact on the business.

That said, just because a particular test organization has a test fail shouldn't automatically make a bug "critical." For example, there might be a temptation to say something like "all system tests must pass before shipping, thus any system test that fails automatically results in a critical/high-severity bug." Hopefully no organization would make that statement (good test exit criteria is a different discussion), but the point is that the "severity" of bug should be judged on its impact to the customer, product, or company image rather than where or when it is uncovered.

One possible way to address that is to make a distinction between "severity" and "urgency." For example, as the release date nears there can be time pressure to determine if an apparently lower-severity bug might affect a large subset of customers, giving the bug a greater "urgency" and elevating work on that bug above (some) others at least until that determination can be made. A proper balance balance between the two, along with experience and good judgement, would help direct where time and effort is spent.