Our software uses in-house written deployment software due to the complexity of the product. As part of our current QA (not dev) build process (using more in-house software) we compile the code, patch the databases, deploy the software and run a smoke test and, in some cases, automated UI tests, before the build is "passed". We are moving to TFS2010 and getting rid of the in-house build process and I would rather not have to add this process into TFS builds.

What processes do you use to ensure that the deployment is successful and reporting errors back to the development team? Should a build be classed as a failure in the event of deployment errors / smoke screen or automated UI test failures?

TFS ends it's build process when the files are placed in the drop folder it then marks that build as OK. If I am to use the same solution as we have now, either TFS build has to be extended to perform all these tasks before marking the build as a success or we have to have a mechanism to mark the build as failed after it has previously succeeded.

  • (1) I still don't understand the question about failure. Failure is failure. What are you asking? (2) The part about TFS is -- what -- a complaint? Or are you asking for help customizing it? It's not clear what you don't already know and what you want to know.
    – S.Lott
    Commented Jul 15, 2011 at 10:02
  • 1) I'm curious if the "TFS build" is classed as failed, or if it is a build has bugs. 2) There's a lot of extension needed to the TFS Build Process, is it needed. Or should the "build" stop at the drop folder, and then another process (one that logs bugs for example) take over afterwards?
    – DaveShaw
    Commented Jul 15, 2011 at 10:16
  • (1) Failed is failed. "I'm curious if the "TFS build" is classed as failed, or if it is a build has bugs"? What hair are you splitting? Please update the question to explain how failed could be "classed" as something other than failed. (2) If you're asking about TFS Extensions, please update the question to clarify what you're asking for advice on. The question is murky and hard to follow.
    – S.Lott
    Commented Jul 15, 2011 at 10:21
  • Scratch this. Looking in more detail at TFS there is a "Build Quality" this is what I should be using to feedback the quality of the build to development.
    – DaveShaw
    Commented Jul 15, 2011 at 10:28
  • What? "Scratch this. Looking in more detail at TFS"? What do you want to know? Are you looking for this? msdn.microsoft.com/en-us/library/ms181734.aspx
    – S.Lott
    Commented Jul 15, 2011 at 10:55

3 Answers 3


I think it comes to too who can fix the problem, if anyone that can check in is expected to be able to fix “errors in the deployment” then it should fail the normal build.

Otherwise I would tent to go for two “builds”, the first one compiles the code and runs the unit tests. The 2nd build takes the output from the first build then does the deployment checks. This way a normal programmer that does not work in the deployment system can keep working if another programmer has expanded the app in a way that needs the installation scripts to be updated.

Failing a build that for a problem that few people can fit tend to lead to failing builds being ignored.

You wish all the developers to stop work until they have fixed the main build, this will only happen if they are always able to fix it.


If the build cannot generate the expected output, it is broken, and the build engine needs to notify those who broke it and anyone else who needs to know.

Note that, if somebody can break a process not relevant to them, you might have stronger dependencies than you like in your build methodology and you may want to investigate into that to get rid of them.


Any error in any compile or test merits marking the build as a failure.

Any errors in the deployment may or may not. It depends on what needs to be done if one occurs. Usually any error means someone somewhere needs to take (corrective) action. So yes, I would mark the build failed. I guess a warning might be ok to if you want to distinguish between build and deployment errors, but I think that is a matter of semantics.

The important thing is to ensure that notifications are sent and - more importantly - read. For example, I dump all mails from our buildserver straight into the trash, unless "failure" or "failed" is somewhere in the subject.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.