For which problems are dependency exclusions sensible solutions?

Question

I've seen a few build tools in my career, and they've all had their quirks. I'm just now looking into Maven, and came across the idea of "dependency exclusions" for the first time. I honestly don't understand which legitimate problem this mechanism is trying to solve.

Let's take the POM of Apache Pig, for example:

<project xmlns="http://maven.apache.org/POM/4.0.0">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.apache.pig</groupId>
    <artifactId>pig</artifactId>
    <packaging>jar</packaging>
    <version>0.16.0.2.5.0.0-1245</version>
    <dependencies>
        <!-- ... -->
        <dependency>
            <groupId>org.apache.avro</groupId>
            <artifactId>avro</artifactId>
            <version>1.7.4</version>
            <exclusions>
                <exclusion>
                    <!--  Don't pull in Avro's (later) version of Jetty. -->
                    <groupId>org.mortbay.jetty</groupId>
                    <artifactId>jetty</artifactId>
                </exclusion>
                <exclusion>
                    <!--
                     Exclude Avro's version of ant since it conflicts with Jetty's.
                    -->
                    <groupId>org.apache.ant</groupId>
                    <artifactId>ant</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <!-- ... -->
    </dependencies>
    <!-- ... -->
</project>

This particular project depends on the avro artifact from the org.apache.avro group, but removes two dependencies from that dependency (namely jetty and ant) because they "conflict". Apparently, it's perfectly fine to remove these dependencies from avro, as the ones included by pig will do.

---

To me, this sounds like asking for trouble, and I'd think that any other resolution of dependency conflicts would be preferred over it.

By meddling with the dependencies of a dependency, the Pig project is effectively patching the Avro project, and quite blindly too, as the version constraint on Avro does not strictly identify one release of the project.

Wouldn't it make more sense to either reject the build altogether, or build in two separate versions of both ant and jetty into the project (assuming they can be kept segregated in Java's class loader system)?

Answer 1

There are two problems that it solves, and both have work-arounds, but neither work-around is always available to real-world projects.

But first, for those who aren't familiar with the Maven dependencies mechanism:

1. Maven allows you to specify the specific versions of your application's dependencies in the POM (project descriptor).
2. Your dependencies may themselves have dependencies, in which case your project will include those dependencies (critical when building a WAR, because otherwise it won't run).
3. As you work your way through the graph of dependencies, you may find multiple dependencies for the same artifact with differing versions.
4. Maven has a somewhat bogus approach to resolving those conflicts, in that the version closest to the root of the tree wins. If there are multiple versions at the same depth of the tree, the resolution is arbitrary: "the first definition wins" but there are no rules for which is first.

This mechanism can sometimes give you dependencies that you don't want, so you use the exclusion rule to prevent Maven from including specific transitive dependencies.

So, on to the problems.

Problem #1: the undesired dependency

Let's say that you're building a web-app using Spring 3.x, and moreover, you are using SLF4J with the Log4J adaptor. However, Spring 3.x uses Commons-Logging internally. No problem, SLF4J also includes an adaptor to replace the Commons-Logging implementation.

However, depending on the specific order that Maven stores transitive dependencies in the WAR, you might find that the Spring logging messages are in fact going through the Commons-Logging implementation. By adding an exclusion to the relevant Spring dependencies, you prevent this from happening.

I would argue that the correct way to resolve this problem is for Spring to mark its dependency as provided. That will prevent Maven from automatically including it in the generated WAR, on the assumption that the main project or execution environment provides the dependency. Unfortunately, you probably won't be able to convince dependency authors to do this (and in many cases it's not the correct approach), so you add exclusions where needed.

Problem #2: incompatible transitive dependencies

This is the case that you appear to have: the version of jetty that's depended-on by avro is not backwards-compatible with the rest of the project. This is generally infrequent for well-used Java artifacts, because backwards compatibility is a positive thing in the Java community. But sometimes bugfixes make that impossible.

This is a case where the application author (ie, you) needs to step in and ensure that all of the dependencies play nicely. If your project requires an older version of jetty then either you need to use a version of avro that supports that version, or far, far better update whatever it is in your application that depends on the old behavior.

An ugly work-around is to explicitly specify the dependency versions in your project, using either a dependency-management section or directly referencing the dependency. Both of these violate the whole premise of transitive dependencies, but ...

Answer 2

It solves the problem of third party libraries that set versions of common dependencies more narrowly than necessary. For example, they may actually work with anything 2.0 or above, but they set their requirements at 2.4+, or worse, exactly 2.4.3. Why would someone do this? They may not have the resources to test on previous versions, and don't feel comfortable claiming support. A lot of times, they just plain don't know any better. Whatever the reason, it's not an uncommon occurrence in the wild.

Obviously, if you have control over both libraries, the best course of action is to fix the dependencies on the other one. If you don't, exclusions let you take the testing risk in order to get everything working together.

And no, multiple versions of a library is not feasible. Even if language runtimes could deal with the namespace clashes, which they can't, you'd still have issues if those two versions actually need to interact. There is no way to resolve conflicts between different versions of data types and code. The only way to use two separate versions of a library is if they are in two separate processes, like in a microservices architecture.