1

In the past when I've set up Jenkins to look for version control changes every second by using the cron format of "* * * * *" (or something like that). When you do this, it gives you a warning and asks if you meant to say something like once a day.

My question is, why would checking once per minute ever be a bad thing? The only bad aspect that comes to mind is that it may be a performance issue. Otherwise, I would want my continuous integration server to always be checking for changes as frequently as possible so that I can know as soon as possible if an error got introduced. Is there a perspective I'm not seeing?

  • 2
    every minute means that it is essentially constantly checking the repo for changes, that is kinda expensive... – ratchet freak Feb 20 '15 at 23:06
  • How many repos do you have? – enderland Feb 20 '15 at 23:07
  • Keep in mind that if you do a big push, it might take more than 1 second for all the objects to reach the remote repository. This could possibly generate some undefined behaviour if Jenkins checks the repository every second. – Radu Murzea Feb 20 '15 at 23:08
  • 5
    @RaduMurzea I sure hope not! Even ordinary manual user interactions are somewhat concurrent on big projects, and the numerous automated tools that may access the repository compound the problem to a degree that "closing your eyes and hoping nobody does anything concurrently" strikes me as irresponsible. DVCS developers are smart. They should manage to make pulls and pushes and other operations atomic. – user7043 Feb 20 '15 at 23:13
  • @ratchetfreak what would you set it to? – Daniel Kaplan Feb 20 '15 at 23:18
5

There are two things here - the client (in this case Jenkins) and the server (svn or git or something else).

As the client, asking every minute can consume a bit of resources - network and disk on the server, not to mention its own internal builds that compete with local resources for checking if anything has been updated.

At the server side, having a half dozen projects that get polled every minute can likewise be consuming in resources (disk and network) and can start to contend with other client updates (like your developers).

The thing is, if you want to start the build as soon as possible, there are other options for this. You can do a push instead of a pull. For example, on Stack Overflow: Jenkins CI: How to trigger builds on SVN commit and How to configure Git post commit hook in which the act of pushing a commit to the server will trigger a Jenkins build. It will never need to poll (much less every minute) to see if it needs to build because the server tells it to do so when it needs to.

As a follup to all this, I would suggest reading Polling must die: triggering Jenkins builds from a git hook which takes a more opinionated take on the issue.

One should look at the reason behind the desire to have the server polling every minute. This is what Jenkins, in part, seeks to address with the warning. If you want "building as soon as a commit happens" then hooks are the best technology to use for both the build server and the source of the data.

On the other hand, on many projects, a delay on a build isn't as big of an issue as a delay of a few minutes would make it out to be. On several of the projects I've worked on a full test and analysis (as done on the CI server because it took too long to run on a developer's machine) took a good 10-15 minutes if the build server wasn't being taxed with other builds at the time (all the unit tests and static code analysis on a 1.5M SLOC project, then fire up a HSQLDB and jetty instance and run another set of tests...). Doing the polling every half hour was more than sufficient (and if a developer wanted the build to start now so that it would be done before he or she wanted to go to lunch or leave for the day and know with confidence they wouldn't incur the wrath of a broken build upon return... well, there is always the "build now" button)

  • 1
    Thank you, I actually didn't know these push options existed for SVN. – Daniel Kaplan Feb 20 '15 at 23:26
  • Also, I think I had an XY Problem. I really should have asked, "I think I should be 'integrating' as soon as possible, is that wrong?" – Daniel Kaplan Feb 20 '15 at 23:27
  • @DanielKaplan ideally, you would be checking in often (rather than large monolithic commits). Possibly, working on another branch and merging from master frequently. Touching on branching, give Advanced SCM branching strategies a read. Look at In centralized VCS, is it always good to update often?, How often should a programmer commit? and Check In Early, Check In Often – user40980 Feb 21 '15 at 1:15
  • Sorry, I was ambiguous. When I said "integrating" I meant, "the continuous integration server should be running as soon as possible after a change occurs to the source code". Would you agree with that? – Daniel Kaplan Feb 21 '15 at 1:17
  • 1
    @DanielKaplan Ideally, yes. Though within the constraints of the technology. Thats why commit hooks work very nicely - it doesn't put extra load on the servers and serves the integration rapidly. Though, a wait of 5-10 minutes (*/10 * * * * *) isn't bad either (given that CI servers sometimes have integration test suites (fire up a jetty instance, build a database, run some tests) that can take 5-10 minutes themselves on large projects). The "I want it compiling now doesn't" have as much reason if it still takes awhile to do the full build and test on the CI server. – user40980 Feb 21 '15 at 1:22
2

I'm not sure the polling resources argument is compelling. It's a relatively efficient operation on most version control systems, especially if you're just checking and not actually downloading the changes yet. Although, a push notification is certainly preferable.

Usually it's the building resources that are the limiting factor, depending how frequently commits happen and how long your builds take. If you start a build too frequently, and your build nodes max out, that can increase the average latency to get results, especially at the busy times. Also, what can happen is a commit breaks the build, but by the time you get the results, you have several later builds broken by the same bug. This conditions developers to pay less attention to the CI server.

So if you happen to have fast builds and your build nodes never max out, by all means increase the polling frequency. If your build nodes are always maxed out and commits always break 3 or 4 builds before you even know it, then decreasing the polling frequency can help.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.