228

Surely, by the time something gets committed to master, a developer has already run all the unit tests before and fixed any errors that might've occurred with their new code. Or not. There can be many reasons why this can happen: The developer doesn't have the discipline to do that They have forgotten They didn't commit everything and pushed an incomplete ...


152

No. The simplest example I've seen is: branch A cleans unused imports in a file. Branch B adds code that actually uses some of the unused imports. Git merges automatically since the lines that were changed were not the same. Code can no longer compile and unit tests can not run.


114

No. As a counter example, consider branch A adds a unit test that uses reflection to check for a misspelling in an enum. And branch B adds a misspelling. Both pass because a misspelling doesn’t fail a build, in A the test doesn’t fail because everything is spelled right, and in B there isn’t a test to check it. There won’t be any merge conflicts because the ...


75

As a developer who doesn't run all the integration and unit tests before making a commit to source control, I'll offer up my defense here. I would have to build, test and verify that an application runs correctly on: Microsoft Windows XP and Vista with Visual Studio 2008 compiler. Microsoft Windows 7 with Visual Studio 2010 compiler. Oh, and the MSI ...


75

Yes, I agree that randomness shouldn't be part of a testing suite. What you want is to mock any real randomness, to create deterministic tests. Even if you genuinely need bulk random data, more than you can be bothered generating by hand, you should generate it randomly once, and then use that (now set in stone) data as the "random" input for your ...


68

CI-driven development is fine! This is a lot better than not running tests and including broken code! However, there are a couple of things to make this easier on everyone involved: Set expectations: Have contribution documentation that explains that CI often finds additional issues, and that these will have to be fixed before a merge. Perhaps explain that ...


60

I would be against doing this for the following reasons: Any time you set up an automated tool to change code on your behalf, there is the risk that it will get it wrong, or that a situation will arise where you need it to stop making that change (e.g., the latest version of Google Mock had a bug in it, so it's not your code failing) and you have to waste ...


56

Your build number won't be reset to 0, when minor and major versions increase, this violates sections 7 and 8 of the specs: Minor version Y (x.Y.z | x > 0) MUST be incremented if new, backwards compatible functionality is introduced to the public API. It MUST be incremented if any public API functionality is marked as deprecated. It MAY be incremented if ...


53

This question is really two questions in one. Todo comments Of all the ways to track action items, this is the worst. TODO comments are good during active work or as a way of suggestion to a maintainer, "here is something that could maybe be improved on in the future". But if you rely on TODO comments for getting work done, you're doomed to fail. What ...


47

The problem I see here is that you have made the code coverage a trigger for build failure. I do believe that code coverage should be something that is routinely reviewed, but as you have experienced, you can have temporary reductions in your pursuit of higher code coverage. In general, build failures should be predictable. The following make good build ...


46

I've worked on projects which use anywhere from no to extensive randomness in tests, and I'm generally in favour of it. The most important thing to remember is that the randomness must be repeatable. In the current project we use pytest-randomly with a seed based on the pipeline run ID in CI, so it's trivial to repeat a failing run identically, even though ...


43

You should look at git-flow. It's an excellent (and popular) branching model. Git Flow Summary Branching The main trunks that stay around forever are develop and master. master holds your latest release and develop holds your latest "stable" development copy. Contributors create feature branches (prefixed with feature/ by convention) off of develop : $...


43

Let me be the one to disagree with my fellow answerers. This is known as Gated Check-ins in the TFS world, and I expect elsewhere. When you attempt to check-in to a branch with the gated check-in, the shelveset is sent off to the server, which makes sure your changes build and the specified (read: all) unit tests pass. If they don't, it notifies you that ...


37

Have you considered not using code coverage metrics? I'm not going to argue that code coverage isn't something that you should look at. It absolutely is. It's good to keep track of what was covered before a build and after a build. It's also good to make sure that you're providing coverage over new and modified lines of code in a change (and, depending on ...


36

No, it's not, for two reasons: Speed Commits should be fast. A commit which takes 500 ms., for example, is too slow and will encourage developers to commit more sparingly. Given that on any project larger than a Hello World, you'll have dozens or hundreds of tests, it will take too much time to run them during pre-commit. Of course, things get worse for ...


36

Here is an example which neither does require changes to the existing tests itself, nor reflection, nor a failing build, for not giving the wrong impression such cases can only happen under artificial circumstances. Assume the codebase contains a private function f which is currently not called anywhere (maybe it was in the past, but noone has deleted it). ...


34

Building a sustainable plugin model requires that your core framework expose a stable interface that plugins can rely on. The golden rule is that you can introduce new interfaces over time but you can never modify an already published interface. If you follow this rule, you can refactor the implementation of the core framework all you want without fear of ...


32

Continuous integration as a term refers to two distinct ideas. The first is a workflow: instead of everyone in a team working on their own branch and then after a couple of weeks of programming try to merge their changes into the mainline, that changes are integrated (nearly) continuously. This allows problems to surface early, and avoids incompatible ...


31

Personally, I choose option 3: keep versioning information in VCS metadata, specifically, tags. Git makes it very easy to do so, because there is a command git describe, which can uniquely describe a commit based on a tag. Here's how it works: If the current commit is tagged, output the name of the tag. Otherwise, walk the history backwards until you find ...


28

Let's agree on terms first. I personally use the terms Continuous Build and Continuous Integration to distinguish two different scenarios: Continuous Build: a tool that checks periodically if the repository changed since the last build, and build/test if it did. Continuous Integration: a tool that takes Pull Requests and validate them against the latest ...


24

True, you do not have particular need of a CI system to perform builds and check that those builds are correct, but that is only part of what CI is about. The purpose of CI is to detect errors as soon as possible, because generally speaking, the earlier an error is caught the cheaper it is to fix. To that end, in the case where a build step is not necessary,...


24

Do not use TODOs. You already have a TODO list in your project. It's called the issue tracker. I think the real problem is in this sentence: we can create a ticket in our issue management system, which creates clutter and also might get moved to a later sprint or the backlog by management. If your issue tracker creates to much clutter, find ways to fix ...


23

Apart from the excellent Oded answer: You test the code from the repository. It may work on your machine with your files... that you forgot to commit. It may depend on a new table that does not have the creation script (In liquibase for example), some configuration data or properties files. You avoid code integration problems. One developer downloads the ...


23

Testing scientific software is difficult, both because of the complex subject matter and because of typical scientific development processes (aka. hack it until it works, which doesn't usually result in a testable design). This is a bit ironic considering that science should be reproducible. What changes compared to “normal” software is not whether tests are ...


22

You'd think so wouldn't you - but developers are human and they sometimes forget. Also, developers often fail to pull the latest code. Their latest tests might run fine then at the point of check-in, someone else commits a breaking change. Your tests may also rely on a local (unchecked-in) resource. Something that your local unit tests wouldn't pick up. ...


22

It sounds like you have a few problems here: 1. Identifying features for a specific release This is a project management issue, and a coordination issue. Will this feature be released before, at the same time as, or after this other feature? If releases want to happen one feature at a time, then identify that. If features are going to be grouped into ...


18

You can mitigate the effect to some degree by allowing the relative code coverage to reduce when the total number of uncovered lines also reduces, or when the total number of lines reduces, since this are pretty clear signs of a refactoring which sets a new base line for your coverage metrics. In your example, the total number of uncovered lines reduces ...


17

Traditionally, before Continuous Integration, Continuous Build, Continuous Deployment, Continuous Testing, etc., the system was split into modules and those modules were developed by independent teams which didn't interact very much. The term "integration" refers to the event when those independently developed modules were assembled together into the ...


16

In General When do you have enough automatic testing to be confident in your continuous integration pipeline? The answer probably becomes clear if you think about what you want to be confident about. Ultimately, it maps 1-1; every test makes you confident about the one thing it tests: Unit testing gives you confidence that a class (or module) does what ...


Only top voted, non community-wiki answers of a minimum length are eligible