0

What are the generally accepted means of running tests that pass on the dev machine on the build server where they'll consistently fail due to missing dependencies? (In this case, a configuration file that is managed by another completely external application.)

My app is reading an INI file containing various key/value pairs, and I'm storing the path to that file in App Settings. This works fine in my dev environment and in production as well.

But what about when I send all of this to TFS for a build? Those tests are going to fail because neither the INI file nor the Path setting pointing to it exist on the server (nor do I think they should, but I may be mistaken with this).

To quote Jon:

Typically a build machine is meant for build, compile + unit tests. There should be no external dependencies.

Should I include logic to just skip those particular tests if they're not running in dev?

--EDIT--

I have three working environments:

  1. Dev (my workstation)
  2. Build/Test (a TFS instance running on my server)
  3. Production (the application running at the customer site)

The app has a UI for specifying the path to the INI file. This path is stored in App Settings, which in turn is stored in a subfolder under %AppData% (in Windows). The file is created and updated by an external application whose source code is not under my control.

Tests succeed in Dev and Prod, but fail in Build due to the fact that neither the INI file nor the Path setting pointing to it exist in Build.

According to my research, external dependencies (like this INI file and its corresponding setting) should not be introduced in a Build environment. I tend to agree. It seems small now with just this one instance, but I can easily foresee this spiraling out of control as more such scenarios are introduced.

It wouldn't take much of this before Build became choked down with all of these specialized, tightly coupled, brittle and difficult-to-maintain configurations.

How do we get out of this Catch-21?

  • You're making me dizzy. I'm hearing: dev=pass, build server=fail, dev=fine, production=fine, TFS=?, candied_orange=huh? Please pick some consistent terms to use and make their meaning and what's going on clear. – candied_orange Aug 6 '18 at 1:27
  • @candied_orange—TFS is Microsoft's build server—among other things not relevant for this topic—thus the tag. Environments: dev=development, prod=production. I'm using what I believe so far is ubiquitous terminology in our world. Lord knows, I don't want you to get dizzy :-) Can I clarify anything else? – InteXX Aug 6 '18 at 1:45
  • 1
    My problem isn't the abbreviations. It's understanding what your problem actually is. I suspect you're saying the same thing different ways and are causing confusion by doing that. Please edit the question to make your situation clear. – candied_orange Aug 6 '18 at 3:11
  • You say your environments are dev and prod. I'm trying to get you to be clear about the other environment. The one in which it's failing. You've mentioned "DevOps remote build", "build server", and "Team Foundation Server". If these are three different ways to talk about the same test environment (that for some mysterious reason you think can't be allowed to have the ini file dev and prod have) please edit the question to make that clear. – candied_orange Aug 6 '18 at 7:39
  • 1
    Why in the hell are your tests touching any INI files? Those should be mocked out. – Euphoric Aug 6 '18 at 8:05
3

Its best to design your program so that it is able to run after it's built.

In this case you need a default path in app settings, pretty easy, and a default ini file to be copied to that path.

Include these things in the project and if necessary add post build steps to move them to the right path or what have you.

Alternatively, you might consider these test Integration tests. In which case, don't run them immediately after build. (Skip them by adding categories to your tests, so you have unit and integration) First deploy the program to your test environment with another automated step (Octopus deploy?) and then run them.

  • I like this way of looking at things. I believe this may be the answer I seek, but I want to wait a bit before accepting it. Do you mind? @candied_orange may want to answer as well. – InteXX Aug 6 '18 at 19:06
  • Octopus deploy? FYI I'm using TFS Release Tasks for deployment. – InteXX Aug 6 '18 at 19:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.