7

Using third party libraries for productivity gains in software development is common. Unfortunately, along with the library's functionality we also import its bugs. Some of them get fixed in subsequent releases. So, to upgrade or not to upgrade, this is the question.

I am interested in learning from experiences when upgrading to a newer version of the library was desirable, but after a cost/benefit analysis the conclusion was that upgrading was not a good solution "in the grand scheme of things". I am interested in finding out what forces influence the decision towards not upgrading.

Improve this question
7
  • 1
    I don't see how you can predict ahead of time whether a newer version of a library will introduce bugs, incompatibilities or other problems, other than to evaluate in a general way the reliability of the vendor. You have to balance the risk of introducing a bug against the opportunity cost of keeping the existing version.
    – Robert Harvey
    Jan 23, 2014 at 18:22
  • seems you already know the answer. It's pretty much what you said: cost benefit analysis. Trade off between benefit from new features vs. (risk of unknown bugs x potential cost of those bugs + cost of known bugs). Depending on how critical the library is to your product, risk of bugs will probably translate into how heavy you need to retest your product, so that's a more tangible cost to your time.
    – DXM
    Jan 23, 2014 at 18:22
  • Usually the decision to not upgrade comes from someone who ran a test with their code using the upgrade and found that it introduced a visible/noticeable/non-trivial problem. Sometimes the decisions to not upgrade comes from a vendor of a different third-party component who has not yet certified their component with the component in question.
    – FrustratedWithFormsDesigner
    Jan 23, 2014 at 18:27
  • @RobertHarvey,thank you for your comment. Reliability in what sense? What would I look at if I were to assess this, other than, say, how many commits have been issued lately to know that it is still maintained?
    – Mircea
    Jan 23, 2014 at 22:37
  • The number of recent commits that a library has had has no bearing on a library's reliability or support. If anything, a well-maintained, stable library should have fewer recent commits, not more. This is one of the most common logical fallacies about libraries: if it hasn't got a recent commit, then it must be dead.
    – Robert Harvey
    Jan 23, 2014 at 22:42

2 Answers 2

Reset to default
8

The short answer is that the new version could introduce new bugs, as Robert Harvey mentioned in the comments.

To be honest, I think you are approaching this the wrong way. In my experience, the better approach is to default to sticking with the version you have. Presumably, your team has already tested the version you use and how it integrates with your system. If not, then your clients have.

Sure, when a new version comes out, your team should see if it addresses any bugs you have encountered, provides features you could take advantage of, or improves overall performance. Only after running it through testing again should you actually start using it.

The problem with approaching upgrades as something which is done by default and only halted if someone finds a problem is that it is too easy to become lax on the vetting process. Eventually, this will bite you when a new version introduces new issues.

If you stick with the version you have, you know that the overall system behavior will not change.

Improve this answer
2
  • 2
    +1 if it ain't broke don't fix it or add something I really want. We've all worked with some "nice to have" tools that if the upgrade causes major problems, you either roll it back or do without.
    – JeffO
    Jan 23, 2014 at 19:05
  • +1, in addition, new versions often mean slight differences in usage (new parameters, new requirements), which may mean additional effort on your part to keep things functional.
    – GrandmasterB
    Jan 23, 2014 at 20:59
4

To answer, let me first turn the question around on the need to upgrade.

If you never upgrade, over time you are stuck on an outdated version and will find bugs in your system that results from bugs in the library that have been fixed in later versions.

Assuming that you are continuing to develop your system, and the relevant library is of any size of complexity, sooner or later it will be worth upgrading. I.e. the cost of retesting justifies the benefit of the remaining on the latest version.

So, you may not need to upgrade if these assumptions fail.

I.e.:

  1. You are not actively your application.
  2. The library has minimal changes, perhaps only changes to specific problems that you know are not relevant to you.

It's more complex that this, as (2) is a moving target. With each library version that comes out, the accumulated change increases and hence the likely value (as well as risk and testing requirements).

Note that if you delay upgrade a long time "because so far none of the fixes matter to me", then when you do upgrade you may find it much harder due to accumulated impact.

Further, you may have multiple third party libraries that may require upgrade and retest from time to time. One approach is to review and consider upgrade of all third party libraries once per year (or whatever period), and execute the upgrade within a single project, allowing a single full system regression test. This might include upgrade to compiler or operating as well.

Finally, any risk in an upgrade will be inverse to the amount of automated testing in place. If you have very little (relevant to the library) and need to do a lot of manual testing you need more benefit to justify the upgrade cost.

So for example, upgrading a user interface library which requires extensive GUI retest is different to upgrading an XML lib which reads in a few files amenable an automated test.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.