4

We build products which often integrate with third party systems. Sometimes these systems error, which in turn causes our system to error. For example, we call a web service when a user updates their data after we've saved to our own database. While testing, the service on the other end is not functioning correctly.

I can see both sides; on one hand the software won't work until the service is fixed, and so we need to track that somehow. On the other, there's nothing we are going to do that will resolve the defect and having a critical defect in our system sitting there seems to be a waste (and may concern others looking at the tracking system wondering why we have defects we're not acting on).

My question is, should these be tracked as defects in our tracking system, or should our QA people not be logging defects if they know this is the reason (typically they do as they'll ask me if unsure)?

Edit: A few people are making assumptions. The web service were calling is developed by another shop, and we are both being paid by the same client to integrate our systems. Our UI handles the error correctly it doesn't show a ysod. It tells the user they cannot change their account at this time. The defect being reported by our qa team is of course that the user ought to be able to change their account, but they don't know if the issue is something we need to fix or the other company. According to our requirements, we cannot save the users changes if the service call fails. Its been decided that these failures should be uncommon enough that this design is ok. Were also working with the other company to iron out some details that have come up during our testing.

|improve this question
2

Your question blurs the line between project management and software development.

How do you define what is and isn't a "defect"?

Are you logging a defect each time the web service fails?

For me, I would put this down as a project issue. You still want to document the cause / research etc. you have put into it. I don't like polluting my defect / bug list.

Each time you have to explain to the customer or user "The defect list has grown by 43 defects this week, but 39 are the third party problem" just makes managing your defect all the more confusing.

|improve this answer
  • The qa team has been asking me how to handle these issues as they know they're testing the first version which has this integration. – Andy May 12 '13 at 20:19
2

I'd say that for the given example (if it is indeed a real world example) you should definitely be filing that as a defect in your own tracking software. The way I read that, the issue is that your system does not cope well with a third party failing. Even if it's a simple mechanism to record the event and inform someone then your system should be able to cope with that scenario.

More generally, I still say that yes, it's a defect in your system and should be tracked as such if your application is mishandling an error situation. It needn't be filed as a critical issue if the third party only disappears once in a blue moon, but if it's a regular occurrence then you could/should handle it.

|improve this answer
  • I've added more detail, our software behaves in this instance as specced and shows an friendly error message. – Andy May 12 '13 at 20:21
  • If it's behaving as spec'ed and handling the error gracefully then no, definitely not a defect to be filed against your software. – chooban May 12 '13 at 21:41
1

Your system is YOUR SYSTEM no matter what 3rd party software/services you are using. Defects should be tracked and reported. If your software isn't working (regardless of the cause) it is still a bug in your software.

When Microsoft releases a new OS and an old program that uses an undocumented, intended only for Microsoft's internal use purposes causes the old program to break. Is it a bug with Microsoft's OS or the old program's code? To the customer, it is a problem with the new OS. Period. Same goes for your software.

EDIT: As for leaving a critical defect that you aren't going to fix sitting in your tracking system. Once the bug is reported and tracked, management could decide not to fix it, to fix it, to come up with a work around or to keep it in the list as a low priority item. If management decides to not fix it then the defect is closed. It won't be sitting there as active. If the defect is critical then I have no idea why on earth you would decide not to fix it, even if the fix is to simply display an error message to the user informing them that a necessary service is currently down.

Also, I think it is a defect in YOUR SYSTEM if you aren't properly handling a "critical" error that you know exists in 3rd party software.

|improve this answer
  • I almost downsized you, but someone else made a similar answer. I've proved more detail which shows why I don't think its as cut and dry as your answer states. – Andy May 12 '13 at 20:15
  • I've worked on systems where we had to coordinate with 6 other companies, each developing their modules, plus including numerous 3rd party libraries. It's still the same. It doesn't matter if someone else is developing the sw or you; it doesn't matter if it's the same customer. If your program crashes by whatever means then you should be finding a graceful way to handle it. Don't depend on the other guy. Like I said, management can do a bare bones solution and get customer concurrence such as displaying a popup box/message to the operator or the customer may say crashing is fine. Handle it. – Dunk May 13 '13 at 18:23
  • Ooops, I just read another comment below where you say your software gracefully handles the error and runs according to spec. Then the issue boils down to who is the "prime" contractor. The prime is responsible for integration (unless the contract says otherwise). If your company is the prime then it goes in your defect system and is assigned to the 3rd party company to resolve. If you aren't the prime then now I agree that it shouldn't go in your defect system. Then it would depend on your management as to whether it should somehow get reported back to the 3rd party company or not. – Dunk May 13 '13 at 18:37
1

should our QA people not be logging defects...

The answer to above is solid NO. Testers should log defects as it's their job; they are responsible for quality issues to be adequately reflected in the issue tracker. Telling them not to log defects is equivalent to telling developers not to code or telling managers not to work with people.

More realistic question you should be asking instead would better be, should QA not be opening new defects if it is a known bug?

If you (or testers) open a "generalized" issue like <Third Party System> is freaking unstable, things will get much simpler. Instead of polluting the bug database with multiple hard to aggregate defects, they will be able to update the known "umbrella" defect with the details of various related incidents (and re-prioritize its importance if needed).

If the issues with third party system are as frequent as you describe, such a list of related incidents will eventually form sufficient evidence that some effort is needed to invest in order to address an underlying problem. When developers and QA raise a consolidated concern about reliability of particular system, supported by the evidence of 20... 30... 100 incidents related to it, it is much easier for management to decide that this is worth further investigation and taking care of.

|improve this answer
  • I agree that the same issue shouldn't be continually opening new defects. However, a blanket "3rd party system is unstable" entry is IMO not a good idea. Generally, a defect report describes how to recreate the problem (if known) or some details on when it occurred. Without that info (like might happen with the blanket statement), then I can see everything just getting blamed on the 3rd party software, when it may not be the 3rd party software at all. If it really is the 3rd party sw then the defect can be closed or put on hold with that reason. Then management/customer can take actions. – Dunk May 13 '13 at 18:30
-2

You need to track everything that degrades the overall performance of your system. Otherwise you're left with "it didn't work, and no one knows why".

|improve this answer
  • 1
    My question is not do we track the issue, its should we track it as a defect in our defect system. – Andy May 10 '13 at 22:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.