I'm designing a new language and the package-management system for it (something like NPM, Cargo, Pip, Gem, Cpan, Cabal, NuGet or the like).
I'm trying to decide what's a good way to handle the versioning of a package when only part of it is updated. I'm also curious to learn what's the best way to solve this problem with existing package managers.
Example of the problem
Let's suppose a package exports some types and functions that many other packages use:
# package: string 1.0
type String {
length: Number,
data: byte*
}
function from_NUL_terminated( data: byte[] ) -> String { ... }
function to_NUL_terminated( string: String ) -> byte[] { ... }
function from_repeated_char( length: Number, char: byte ) -> String { ... }
function get_UTF8_length( string: String ) -> Number { ... }
# ...
At some point, something in the "string" package needs to be changed. For instance a bug in from_NUL_terminated()
gets fixed, or a new function get_UTF16_length()
is added.
Because of this change, a new version of the "string" package is released: string 1.1.
This makes everything incompatible between the two versions: the language considers String
from "string 1.0" a different type from String
from "string 1.1". Even though the String
type didn't change: only other stuff in that package did.
A consequence is that packages that depend on a different version of "string" cannot pass String
s to each other. A little function that is only used internally by a few packages causes a major split in the ecosystem of the language.
Obviously this problem must be avoided. How?
Further thoughts about the problem
From the example above it sounds like only types need to be protected from this issue. That's practical in most cases, but it's not technically correct. Ideally functions that don't change shouldn't be re-released in the new version either.
It sounds like every item in a package should have its own versioning, rather than package itself: every type, every function, every piece of data or metada.
What's the solution?
How do existing languages and their package managers handle this situation?
And how could I handle it for my new language and its new package manager? Can the package manager fix this problem on its own for any language, or would it need support from the language to do things properly?
Addendum: a minimal example showing the problem in Node.js
I created a project in Rust where two different versions of the crate "mystr" are used by the "main" crate and the "printerlib" crate: https://github.com/BlueNebulaDev/rust-version-test . (a crate is a package in Rust terminology)
The "mystr" crate exposes a type MyStr
. This type is defined identically in both version. "mystr 2.0.0" also exposes a function that wasn't available in the previous version: mystr::from_slice()
.
The program can't compile, because the "main" package creates an mystr(1.0.0)::MyStr
object and tries to pass it to a function that expects a mystr(2.0.0)::MyStr
object. However MyStr
never changes in the two versions: it's only other functions in the same crate that change.
I also created a Node.js project showing the same issue in that environment: https://github.com/BlueNebulaDev/node-version-test/ .
get_UTF8_length()
needs to be removed from the string package. That's an "incompatible API change", thus it requires a MAJOR version. Nevertheless,String
was not touched.