NuGet Hell - by design our NuGet repo doesn't yet include the new functionality we reference

Question

We have a solution called 'MySolution'. We have another solution which tests this solution called 'MyTests' which tests 'MySolution'.

When in debug mode, 'MyTests' reference 'Mysolution' with a path to local DLLs.

When in release mode, 'MyTests' reference 'Mysolution' via NuGet.

Only when feature branches of 'MySolution' are merged into 'dev' does the build server push to NuGet.

So, problem:

During development I update MySolution with new functionality, I update MyTests to test this new functionality and it works and builds and passes and all is good because it's referencing the locall DLL of MySolution.

When I push to the build server, it fails to build since the new functionality is not found in the MySolution package (now being referenced by NuGet). This fails in exactly the same way if I flick my Visual Studio onto release mode.

What approaches are available to the team to avoid this form of DLL/NuGet hell? Is there a tried and tested method involving something clever with feature branch names (i.e. MySolution feature branches will push to NuGet but be appended with a version that the MyTests feature branch will by default look to)?

Answer 1

I am not an expert on NuGet and have only limited experience with "nuget.exe", but I am pretty sure I understood what your actual problem is, and I think the following may be a conceptual solution:

On your build server, MySolution exists in two versions in parallel:

• the last commited version

• the "NuGet" version (which corresponds to the last time a feature branch was merged into 'dev', as you wrote, which is usually older).

(I assume the concept of updating the NuGet package at that time, and not at every commit was implemented intentionally, and you don't want to change it.)

Unfortunately, your tests exist currently only in one version:

• a version which corresponds to the last commited version of MySolution

And since this version references the NuGet version, these two do not fit together any more.

The solution to this is to create two versions/variants of the tests as well:

• one version which corresponds to the last commited version of MySolution and references exactly that one and nothing else, (so especially in the "Release" configuration, by referencing the "Release" mode DLLs of MySolution).

• one version which corresponds to the NuGet version of MySolution, and references that one via NuGet. This version should be updated exactly at the point in time, and only at that point in time, when the NuGet package will be updated, i.e. when a feature branch is merged into the 'dev' branch.

So I would recommend to setup these two variants of the test project. In the first one you actually work and extend the tests. The second one should be updated by some automatism from the former, but only and precisely at the point in time when the NuGet package is updated. For example, there could be copy script which simply copies the source code files from the first folder over to the second, whilst the project file(s) of the tests are patched to replace the DLL reference by the NuGet reference. There are some other solutions possible, depending on the precise tooling, but I hope you get the idea.

Now call this "copy script" exactly when you merge the feature branch into dev. Depending on your tooling, it may also be possible to make the script do both steps in one: copying the tests and merging the feature branch.

You (hopefully) end up with two test sets:

• one which tests your latest commited development version (and could exists in a debug as well as a release configuration as well, if you need both)

• one which tests you latest NuGet package.

Both can (and should) be executed on the build server.

In short, "the push to NuGet" operation requires a corresponding "push regular tests to NuGet tests" operation, that should solve your issues.

Answer 2

If you are going to test this way you are going to need to be more nuanced about it.

I'm going to presume that the actual library being released to nuget must pass these tests that you are releasing as some form of acceptance strategy.

If that is the case:

1. the other team builds, and tests their nuget package.
2. given they are happy with it they release it to a dev-build repository.
3. this package will have a list of "features" (just a list of labels) that identify sets of tests in the test package.
4. If running all of these labelled tests passes, then the package is promoted from dev-build to prod (or the next stage).
   • To prevent them from running ahead of your tests a minimum code coverage metric should be used so that untested behaviour doesn't just escape to the wild.

When you are writing new tests you will have to write them to a new label. That way they are not run on packages that do not support them. You may also have to break the tests themselves across several binaries if the interface language is incompatible, etc...

This way you can write new tests without unduly restricting release activities.

• If that team hasn't implemented the new feature yet (or defect fix, etc) they can still push through other changes.
• If you haven't yet drawn up the tests for a new feature, the code coverage metric or lack of tests matching that label will prevent their package from going through.