How to handle artifacts across multiple teams and repositories?

Question

Our situation

We have 8 teams and each team produces 8 artifacts (A,B,C, etc). Half of the teams use Git, the rest use clearcase. I would like to have each team utilize Git but this will be a struggle (wish me luck).

Each team is responsible for building their own artifacts. These artifacts can be scripts, iso's, jars etc. All these artifacts are put together to create the product.

Our Problem

Currently we have No System to handle these artifacts. We have each team place on a drive then deliver it to the a person. This person then organizes the artifacts to create the releasable product.

As you can imagine this causes a lot of issues. Its difficult to track versions among artifacts and the releasable product. Human error. etc

Our Plan So Far

We are currently migrating each team to Microsoft TFS. On TFS each team would create their own Build definition and build their artifact. The test team will test each artifact and determine whether they pass or fail.

Holes in the Plan

We are struggling to find a solution that would store and version each of these artifacts and manage dependencies. What we desire is to create the product on the fly from tested versions of each artifact.

Questions and Advice

What solutions would best handle these requirements ? What are some of the best practices and processes for configuration management ?

We experimented with Nexus Repository OSS but we were not happy with some of its functionality.

I believe Git-LFS could be apart of the solution. It enables versioning through tagging, and I believe you can retrieve commits based on the tag. The one downside is, our company has limited Git knowledge (we are getting better) but are not there yet.

Thanks for the help!

If you need more information please let me know. Its hard to describe everything detail without writing a novel.

Answer 1

It sounds like you need a package management system. There's several available depending on the platform you are working with (NPM, Maven, Nuget, Chocolate, etc.)

TFS has built in support for hosting Nuget packages, and Visual Studio 2017 makes creating Nuget packages for .Net Standard and .Net Core apps very easy. However, if you have a heterogeneous environment you will probably need to look at Nexus or Artifactory which support multiple package management systems.

Package Management has both server side and client side features that can help your situation:

• All provided packages have an archive and a manifest.
• The manifest lists all the dependencies for that package.
• The client side (Nuget, Maven, etc.) looks at the current project's dependencies and downloads the dependencies (and their dependencies) so that you can build and test the project.

It's pretty standard for DevOps shops to host their own private repositories, both to manage dependencies and to host internal artifacts that are not meant to be shared with the world.

Nexus and Artifactory both support multiple build systems, so they can play the repository role for more than just one build tool. TFS, Jenkins, TeamCity, etc. all play well with the package management systems, and can even deploy the build artifacts to the package management server automatically.

---

Part of the solution is also having build numbers in your version number. You can include the GIT commit number as the final part of the version number quartet to help you trace it back to a specific commit. Your automated build system can inject that into the package creation step, which can help with bug reports, etc.