225

Surely, by the time something gets committed to master, a developer has already run all the unit tests before and fixed any errors that might've occurred with their new code. Or not. There can be many reasons why this can happen: The developer doesn't have the discipline to do that They have forgotten They didn't commit everything and pushed an incomplete ...


103

With 65 billion tests, it sounds like you're being asked to test all possible inputs. This is not useful--you'd essentially be testing that your processor functions correctly, not that your code is correct. You should be testing equivalence classes instead. This will drastically reduce your range of test inputs. Also consider whether you can subdivide your ...


87

It depends to some extent on how your team usually works, but I would say that was fine. Keeping the build working saves everyone else time. It's polite for the the second programmer to drop the first an email to explain what he has done, just in case a specific version of the library is needed or there is some other complication. It's also a slightly ...


75

As a developer who doesn't run all the integration and unit tests before making a commit to source control, I'll offer up my defense here. I would have to build, test and verify that an application runs correctly on: Microsoft Windows XP and Vista with Visual Studio 2008 compiler. Microsoft Windows 7 with Visual Studio 2010 compiler. Oh, and the MSI ...


68

CI-driven development is fine! This is a lot better than not running tests and including broken code! However, there are a couple of things to make this easier on everyone involved: Set expectations: Have contribution documentation that explains that CI often finds additional issues, and that these will have to be fixed before a merge. Perhaps explain that ...


56

I would be against doing this for the following reasons: Any time you set up an automated tool to change code on your behalf, there is the risk that it will get it wrong, or that a situation will arise where you need it to stop making that change (e.g., the latest version of Google Mock had a bug in it, so it's not your code failing) and you have to waste ...


54

To start with, this comment: ... having a branch implies an extra complexity and thus extra work ... is wholly false. I often hear it from people who aren't accustomed to branching, but it's still wrong. If you have many developers accumulating changes locally, their local changes constitute a de-facto branch of the main repository. When they finally ...


53

This question is really two questions in one. Todo comments Of all the ways to track action items, this is the worst. TODO comments are good during active work or as a way of suggestion to a maintainer, "here is something that could maybe be improved on in the future". But if you rely on TODO comments for getting work done, you're doomed to fail. What ...


47

The problem I see here is that you have made the code coverage a trigger for build failure. I do believe that code coverage should be something that is routinely reviewed, but as you have experienced, you can have temporary reductions in your pursuit of higher code coverage. In general, build failures should be predictable. The following make good build ...


45

Your build number won't be reset to 0, when minor and major versions increase, this violates sections 7 and 8 of the specs: Minor version Y (x.Y.z | x > 0) MUST be incremented if new, backwards compatible functionality is introduced to the public API. It MUST be incremented if any public API functionality is marked as deprecated. It MAY be incremented if ...


42

You should look at git-flow. It's an excellent (and popular) branching model. Git Flow Summary Branching The main trunks that stay around forever are develop and master. master holds your latest release and develop holds your latest "stable" development copy. Contributors create feature branches (prefixed with feature/ by convention) off of develop : $...


41

Let me be the one to disagree with my fellow answerers. This is known as Gated Check-ins in the TFS world, and I expect elsewhere. When you attempt to check-in to a branch with the gated check-in, the shelveset is sent off to the server, which makes sure your changes build and the specified (read: all) unit tests pass. If they don't, it notifies you that ...


39

If this is a real test suite, then you don't want to get anywhere near working on it. The whole job of a tester is to strike a balance between testing thoroughly enough to be confident you've got the "right" results and writing few enough tests that they can be run in a reasonable amount of time. Many tests can be abstracted into "equivalence classes", ...


37

Have you considered not using code coverage metrics? I'm not going to argue that code coverage isn't something that you should look at. It absolutely is. It's good to keep track of what was covered before a build and after a build. It's also good to make sure that you're providing coverage over new and modified lines of code in a change (and, depending on ...


35

No, it's not, for two reasons: Speed Commits should be fast. A commit which takes 500 ms., for example, is too slow and will encourage developers to commit more sparingly. Given that on any project larger than a Hello World, you'll have dozens or hundreds of tests, it will take too much time to run them during pre-commit. Of course, things get worse for ...


34

Building a sustainable plugin model requires that your core framework expose a stable interface that plugins can rely on. The golden rule is that you can introduce new interfaces over time but you can never modify an already published interface. If you follow this rule, you can refactor the implementation of the core framework all you want without fear of ...


33

I guess for your marketing department it is not important how CI works, but what CI means for new releases of your software. CI will ideally mean that you can produce a new potentially releaseable version of your software every day, ready to be presented or sold to your customer, with some new features, functionality or bugfixes added. That does not mean ...


32

Fail fast is a good principle - the sooner you know the build is broken, the sooner the offending commit can be identified and the build fixed. Build on every commit is the right thing to do. Building every 15 minutes can be pointless if the project has a high volume of commits within such a timeframe - tracking down the bad commit would take longer and ...


32

Continuous integration as a term refers to two distinct ideas. The first is a workflow: instead of everyone in a team working on their own branch and then after a couple of weeks of programming try to merge their changes into the mainline, that changes are integrated (nearly) continuously. This allows problems to surface early, and avoids incompatible ...


28

my years of software development experience suggest that in practice it can't work. Have you tried it? Dave and I wrote the book based on many collective years of experience, both of ourselves and of other senior people in ThoughtWorks, actually doing the things we discuss. Nothing in the book is speculative. Everything we discuss has been tried and tested ...


28

Personally, I choose option 3: keep versioning information in VCS metadata, specifically, tags. Git makes it very easy to do so, because there is a command git describe, which can uniquely describe a commit based on a tag. Here's how it works: If the current commit is tagged, output the name of the tag. Otherwise, walk the history backwards until you find ...


27

When someone changes the files that make up the software product and then attempts to check them in (in other words, attempts to integrate the changes into the main product code) you want to make sure that the software product can still be successfully built. There is usually an external system, called the CI server, that either periodically or on every ...


26

Disclaimer: I work for Atlassian DVCS does not discourage Continuous Integration as long as the developer pushes remotely on a regular basis to their own branch and the CI server is setup so that it builds the known active branches. Traditionally there are two problems with DVCS and CI: Uncertainty of integration state - unless the developer has been ...


26

Let's agree on terms first. I personally use the terms Continuous Build and Continuous Integration to distinguish two different scenarios: Continuous Build: a tool that checks periodically if the repository changed since the last build, and build/test if it did. Continuous Integration: a tool that takes Pull Requests and validate them against the latest ...


25

Don't worry about comparisons. Start with Jenkins; it is hugely popular and extremely easy to use. Once you've used it a while you'll learn what features are important to you and what are not. My guess is, you'll end up sticking with Jenkins. I'm sure people will debate whether or not it's the best CI server. Don't listen to them because it doesn't matter. ...


24

True, you do not have particular need of a CI system to perform builds and check that those builds are correct, but that is only part of what CI is about. The purpose of CI is to detect errors as soon as possible, because generally speaking, the earlier an error is caught the cheaper it is to fix. To that end, in the case where a build step is not necessary,...


24

Do not use TODOs. You already have a TODO list in your project. It's called the issue tracker. I think the real problem is in this sentence: we can create a ticket in our issue management system, which creates clutter and also might get moved to a later sprint or the backlog by management. If your issue tracker creates to much clutter, find ways to fix ...


23

More than likely, you arrived at your figure of 65 billion tests by calculating all possible combinations of inputs into the system under test, or by computing the cyclomatic complexity and assuming a test must be written for each of these unique execution paths. This is not how real tests are written, because as other posters and commenters have indicated, ...


23

Apart from the excellent Oded answer: You test the code from the repository. It may work on your machine with your files... that you forgot to commit. It may depend on a new table that does not have the creation script (In liquibase for example), some configuration data or properties files. You avoid code integration problems. One developer downloads the ...


Only top voted, non community-wiki answers of a minimum length are eligible