How to version a project with dependencies and data sources

Question

My apologies if my question is not gonna be clear as I do not have experience in the domain of versioning products/projects. And starting: I have a project which has multiple libraries and data sources dependencies and each library depends to another library or data source. For example:

   <-- L1 <-- L2 <-- DS1
P1                    |
   <--------- L3 <----|

Where P1 is the project, L1-3 is libraries and DS1 is a data source. DS1 has no functionallity, just adding or removing data from it. The final output of the project P1 is a data table. In a point of view the project is not a software but data.

What I want is to use a 'versioning system' which incorporates the changes in each library or data source.

And here are my thoughts.Initially I thought of the following:

• P1 will have an alpha,'beta` etc versioning ( which is always dependand of the versions of libraries and data sources).
• libraries will have a semantic like versioning and
• data sources just an incremental number(etc 1,2,3 etc)

And this would be released internally as:

 --------------------------------------
|project Ver|  L1  |  L2  |  L3  | DS1 |
|--------------------------------------|
|   A       |  1.0 |  1.1 |  1.3 | 1   |
|   B       |  1.2 |  1.2 |  1.4 | 1   |
|   C       |  1.4 |  1.3 |  1.5 | 1   |
|   D       |  1.5 |  1.5 |  2.8 | 2   |
                   .
                   .
                   .

But As i mentioned before some libraries are dependent to some other libraries or data sources and these information are not in that table. How can incorporate it?

If a data source change version (change of the data) should the dependand version change or remain the same?

Gennerally, how would you aproach this problem? Any insights are useful.

Answer 1

Since the real objective of the project is the output data table, I'm inclined to add yet another version number for the output itself. Let's call that OUTPUT.

So now we have all of these things being versioned:

+--------+-------+-------+-------+-------+-----+
| OUTPUT |  P1   |  L1   |  L2   |  L3   | DS1 |
+--------+-------+-------+-------+-------+-----+

Where

• OUTPUT is the data table output
• P1 is the main program
• L1 - L3 are libraries
• DS1 is the data source

Semantic versioning should be a big help. You can "bubble up" compatibility changes ("major" version changes in semantic terminology) where either the output or the dependencies may be affected.

As you've already concluded, semantic versioning is pointless for the data source (DT1), since all changes are "compatibility changes" in terms of the output.

Here's how your (surprisingly fast) project history might go:

+-----+--------+-------+-------+-------+-------+-----+
|     | OUTPUT |  P1   |  L1   |  L2   |  L3   | DS1 |
+-----+--------+-------+-------+-------+-------+-----+
| Sun |      1 | 1.0.0 | 1.0.0 | 1.0.0 | 1.0.0 |   1 |
| Mon |      1 | 1.0.1 | 1.0.0 | 1.0.0 | 1.0.0 |   1 |
| Tue |      1 | 1.0.1 | 1.3.0 | 1.0.0 | 1.0.0 |   1 |
| Wed |      1 | 1.1.0 | 2.0.0 | 1.0.0 | 1.0.0 |   1 |
| Thu |      2 | 1.1.0 | 2.0.0 | 1.0.0 | 1.0.0 |   2 |
| Fri |      3 | 2.0.0 | 2.0.0 | 1.0.0 | 1.0.0 |   2 |
| Sat |      3 | 2.0.0 | 2.0.0 | 1.1.0 | 2.0.0 |   2 |
+-----+--------+-------+-------+-------+-------+-----+

• On Sunday, you've started the project and all versions are at 1 or 1.0.0.
• On Monday, the main project P1 gets a bug fix and the version is updated to 1.0.1. No other version changes are needed. You've programmatically confirmed that this bug fix has not affected OUTPUT.
• On Tuesday, library L1 gets a minor change. It is not a compatibility change, so the version of P1 and OUTPUT will not need changing.
• On Wednesday, library L1 gets a major change. Because of this, your main project P1 needs to be changed as well. However, you've programmatically confirmed that OUTPUT has not been affected, so its version does not change.
• On Thursday, the data source DS1 has been updated. Now the version of OUTPUT will change as well.
• On Friday, the main application P1 has had a major change, which also affects OUTPUT.
• On Saturday, library L1 gets a major change. Library L2 depends on library L1, so it gets a minor change. Nothing else is affected by this change.

I believe that to do this properly will require great discipline. I would recommend maintaining a table much like this one (with actual dates replacing the days of the week, of course) along with very clear notes about dependencies - especially inter-library dependencies!

Every change should be examined carefully for side-effects which "bubble up" through the dependency graph. Having a table should greatly help with this.

Any compatibility change that affects a dependency must update the major version number for this scheme to have a hope of working. A library might get up to version 102.0.0. That's okay, there are plenty of numbers. :-)