Skip to main content
The 2024 Developer Survey results are live! See the results
156 votes

If two individual branches pass unit tests, once they're merged, is the result also guaranteed to pass unit tests?

No. The simplest example I've seen is: branch A cleans unused imports in a file. Branch B adds code that actually uses some of the unused imports. Git merges automatically since the lines that were ...
Esben Skov Pedersen's user avatar
114 votes

If two individual branches pass unit tests, once they're merged, is the result also guaranteed to pass unit tests?

No. As a counter example, consider branch A adds a unit test that uses reflection to check for a misspelling in an enum. And branch B adds a misspelling. Both pass because a misspelling doesn’t fail a ...
Telastyn's user avatar
  • 109k
80 votes

Is the usage of random values in unit testing a good practice?

Yes, I agree that randomness shouldn't be part of a testing suite. What you want is to mock any real randomness, to create deterministic tests. Even if you genuinely need bulk random data, more than ...
Flater's user avatar
  • 53.1k
68 votes
Accepted

How do we avoid CI-driven development...?

CI-driven development is fine! This is a lot better than not running tests and including broken code! However, there are a couple of things to make this easier on everyone involved: Set expectations: ...
amon's user avatar
  • 135k
54 votes

TODO comments with deadlines?

This question is really two questions in one. Todo comments Of all the ways to track action items, this is the worst. TODO comments are good during active work or as a way of suggestion to a ...
Brandon's user avatar
  • 4,575
53 votes

Is the usage of random values in unit testing a good practice?

I've worked on projects which use anywhere from no to extensive randomness in tests, and I'm generally in favour of it. The most important thing to remember is that the randomness must be repeatable. ...
l0b0's user avatar
  • 11.5k
47 votes

Unexpected Code Coverage Reduction

The problem I see here is that you have made the code coverage a trigger for build failure. I do believe that code coverage should be something that is routinely reviewed, but as you have experienced,...
Berin Loritsch's user avatar
37 votes

Unexpected Code Coverage Reduction

Have you considered not using code coverage metrics? I'm not going to argue that code coverage isn't something that you should look at. It absolutely is. It's good to keep track of what was covered ...
Thomas Owens's user avatar
  • 83.5k
36 votes

If two individual branches pass unit tests, once they're merged, is the result also guaranteed to pass unit tests?

Here is an example which neither does require changes to the existing tests itself, nor reflection, nor a failing build, for not giving the wrong impression such cases can only happen under artificial ...
Doc Brown's user avatar
  • 210k
34 votes

How do we avoid CI-driven development...?

Building a sustainable plugin model requires that your core framework expose a stable interface that plugins can rely on. The golden rule is that you can introduce new interfaces over time but you can ...
casablanca's user avatar
  • 4,964
27 votes

Which comes first: CD/Trunk-based development or microservices?

Consensus is you want CI/CD first and anyway, independent of your application's language, design or architecture. Whether you deliver trunk-based or use feature branching is also independent of CI/CD. ...
Martin Maat's user avatar
  • 18.5k
24 votes

TODO comments with deadlines?

Do not use TODOs. You already have a TODO list in your project. It's called the issue tracker. I think the real problem is in this sentence: we can create a ticket in our issue management system, ...
Jens Schauder's user avatar
24 votes
Accepted

continuous integration for scientific software

Testing scientific software is difficult, both because of the complex subject matter and because of typical scientific development processes (aka. hack it until it works, which doesn't usually result ...
amon's user avatar
  • 135k
22 votes
Accepted

Keep branches from piling up

It sounds like you have a few problems here: 1. Identifying features for a specific release This is a project management issue, and a coordination issue. Will this feature be released before, at the ...
Jen's user avatar
  • 693
18 votes
Accepted

Unexpected Code Coverage Reduction

You can mitigate the effect to some degree by allowing the relative code coverage to reduce when the total number of uncovered lines also reduces, or when the total number of lines reduces, since this ...
Doc Brown's user avatar
  • 210k
17 votes

What does "continuous" mean in "continuous deployment", "continuous delivery" and "continuous integration"?

Traditionally, before Continuous Integration, Continuous Build, Continuous Deployment, Continuous Testing, etc., the system was split into modules and those modules were developed by independent teams ...
Jörg W Mittag's user avatar
16 votes

When do you have enough automatic testing to be confident in your continuous integration pipeline?

In General When do you have enough automatic testing to be confident in your continuous integration pipeline? The answer probably becomes clear if you think about what you want to be confident ...
R. Schmitz's user avatar
  • 2,608
15 votes

Unexpected Code Coverage Reduction

This is called Simpson’s paradox, and is a well known statistical issue with your approach. You could even construct cases where you refactor and afterwards every single method has a higher coverage, ...
Aganju's user avatar
  • 1,463
12 votes
Accepted

Where to push a failing test?

What I'd do in this situation is to mark the failing tests as "ignored" - that way you still have the test so that you know what you need to fix in future, but you're not going to end up with broken ...
Philip Kendall's user avatar
12 votes
Accepted

when should unit tests be done in a continuous integration environment

Usually the developer runs them manually before checking in, and a CI server runs them automatically after checking in. Programmers are usually pretty good about running unit tests for incremental ...
Karl Bielefeldt's user avatar
12 votes
Accepted

Should testers have root access or not, on personal VM, given that developers do?

The situation looks pretty simple to me: people need the tools to do their job correctly. If they need root access, they should be given root access and they should be trusted (and trained, if ...
Bryan Oakley's user avatar
  • 25.3k
12 votes
Accepted

How is continuous integration organized in large companies?

It is, basically, a scaling problem. You separate your work into modules, which can be different projects and/or different functionalities of your product. You would have teams that cover sets of ...
Vladimir Stokic's user avatar
12 votes

If two individual branches pass unit tests, once they're merged, is the result also guaranteed to pass unit tests?

Approaching this from a different angle, there's a simple process to ensure that the tests continue passing after merging both branches: a branch must pass CI after being applied to the current target ...
l0b0's user avatar
  • 11.5k
12 votes

Is the usage of random values in unit testing a good practice?

No. Random values in unit tests cause them to be not repeatable. As soon as one test will pass and another will fail without any change, people lose confidence in them, undermining their value. ...
Telastyn's user avatar
  • 109k
12 votes

Which comes first: CD/Trunk-based development or microservices?

Trunk based development with continuous develivery and microservices are mostly orthogonal concepts. However, for a huge monolith, the delivery cycles which usually require a certain amount of testing ...
Doc Brown's user avatar
  • 210k
11 votes
Accepted

Where do I run my integration tests

You'd run them on the test/acceptance environment. The acceptance environment is supposed to mimic production (as much as possible). The only differences should be configuration and possibly amount ...
Mark Walsh's user avatar
10 votes

How to rollback rejected features by QA in a Continuous Delivery scenario?

When a code change A is rejected, don't think necessarily in terms of "rollback feature A by undoing the merge using SCCS". Think in terms of "adding a new change to the code which fixes the defects ...
Doc Brown's user avatar
  • 210k
9 votes

TODO comments with deadlines?

What I'm looking for is a way that a TODO comment can have a deadline against it, and our Continuous Integration system (current undecided which we'll use) would reject the build if this deadline was ...
candied_orange's user avatar
9 votes
Accepted

Why the devops pipeline reads "Code, Build, Integrate, Test"..why build is second?

I write some code, then I build it (compile it) then I run the tests. This happens on my local machine. That's the "code & build" phase in that diagram. Then I check my code in to a central ...
David Arno's user avatar
  • 39.4k
9 votes

If two individual branches pass unit tests, once they're merged, is the result also guaranteed to pass unit tests?

If two individual branches pass unit tests, once they're merged, is the result also guaranteed to pass unit tests? Taking the question at face value, it's very simple to create an example where one ...
Flater's user avatar
  • 53.1k

Only top scored, non community-wiki answers of a minimum length are eligible