This is a bunch of notes around dealing with dependencies in Maven based projects.
Background Info
Before we start here are some useful background info.
SemVer
SemVer is a versioning standard which should help make clear when a package update is safe to use, this is why the below has -DallowMajorUpdates=false to limit to minor or patch updates as documented by SemVer
- Patch version Z (x.y.Z | x > 0) MUST be incremented if only backward compatible bug fixes are introduced. A bug fix is defined as an internal change that fixes incorrect behaviour.
- Minor version Y (x.Y.z | x > 0) MUST be incremented if new, backward compatible functionality is introduced to the public API. It MUST be incremented if any public API functionality is marked as deprecated. It MAY be incremented if substantial new functionality or improvements are introduced within the private code. Furthermore, it MAY include patch level changes. Patch version MUST be reset to 0 when minor version is incremented. So public API calls should be fine with minor and patch releases, with patch releases going even further to being fully backwards compatible. Major releases though maybe incompatible
Major version X (X.y.z | X > 0) MUST be incremented if any backward incompatible changes are introduced to the public API. It MAY also include minor and patch level changes. Patch and minor versions MUST be reset to 0 when major version is incremented.
Maven Nearest Definition Wins
Maven has a strategy to pick which dependency it will use where there are multiple dependencies of different versions are found. In this it does not pick the newest version based on the version number, but rather it will pick the one closet to the project. For example if your project A has dependency B and E directly added and both of them use a dependency called D, but B uses it via C and E uses it directly, then the version used by E will win, regardless of version number. Visualised, in this scenario you will get 1.0 of package D loaded, even though 2.0 is newer. This will could cause the code in C to fail, if it uses anything new or changed in 2.0. If it uses only things from version 1.0 then it would be fine… but any update to B or C in the future could suddenly cause a failure.
A
├── B
│ └── C
│ └── D 2.0
└── E
└── D 1.0
When multiple have the shortest path, it picks the first path in the pom.xml, for example here F and B both bring in D and the path is the same length (2 steps).
A
├── com.demo.F
│ └── D 2.0
└── org.other.B
└── D 1.0
If the pom.xml had this order
<dependencies>
<dependency>
<groupId>com.demo</groupId>
<artifactId>F</artifactId>
</dependency>
<dependency>
<groupId>org.other</groupId>
<artifactId>B</artifactId>
</dependency>
It would load version 2.0 of D because package F was first in the file.
Check for new releases
The following command is a good way to see what is available to update. It takes an absolute age the first time you run it
mvn versions:display-dependency-updates -DprocessDependencies=true -DprocessDependencyManagement=false -DallowMajorUpdates=false
Avoid transitive dependency issues
The big risk in an update is the transitive dependency, for example if com.demo:package:0.1.0 and org.other:package:0.2.0 both use com.demo:depoendency:0.1.0 internally - that dependency package is a transitive dependency of both. It will work since they both use the same version. It can cause issues when doing updates when one is updated and another not, for example if we switched to com.demo:package:0.2.0 which uses com.demo:depoendency:0.2.0 internally but org.other:package:0.2.0 still is looking for the 0.1.0 release it MAY cause issues. It really depends on what the second package uses, in some cases it could be a problem and in others not.
To illustrate this lets look at some real examples.
Example: GRPC
io.grpc:grpc-api:jar:1.70.0 is used by com.google.cloud:google-cloud-spanner:jar:6.93.0 and io.grpc:grpc-okhttp:jar:1.70.0:compile If we were to upgrade io.grpc:grpc-okhttp to 1.82.1 it would break the Spanner code because the GRPC must always be the exact same.
mvn dependency:tree -Dincludes=<package you looking for> can help find these, for example
mvn dependency:tree -Dincludes=io.grpc
...
[INFO] com.demo:package:jar:1.0.0
[INFO] +- com.google.cloud:google-cloud-spanner:jar:6.93.0:compile
[INFO] | +- io.grpc:grpc-api:jar:1.70.0:compile
[INFO] | +- io.grpc:grpc-auth:jar:1.70.0:compile
[INFO] | +- io.grpc:grpc-inprocess:jar:1.70.0:compile
[INFO] | +- io.grpc:grpc-core:jar:1.70.0:compile
[INFO] | +- io.grpc:grpc-context:jar:1.70.0:compile
[INFO] | +- io.grpc:grpc-util:jar:1.70.0:runtime
[INFO] | +- io.grpc:grpc-protobuf:jar:1.70.0:compile
[INFO] | +- io.grpc:grpc-protobuf-lite:jar:1.70.0:runtime
[INFO] | +- io.grpc:grpc-stub:jar:1.70.0:compile
[INFO] | +- io.grpc:grpc-opentelemetry:jar:1.70.0:compile
[INFO] | +- io.grpc:grpc-grpclb:jar:1.70.0:compile
[INFO] | +- io.grpc:grpc-xds:jar:1.70.0:runtime
[INFO] | +- io.grpc:grpc-services:jar:1.70.0:runtime
[INFO] | +- io.grpc:grpc-alts:jar:1.70.0:compile
[INFO] | +- io.grpc:grpc-googleapis:jar:1.70.0:runtime
[INFO] | \- io.grpc:grpc-rls:jar:1.70.0:runtime
[INFO] \- io.grpc:grpc-okhttp:jar:1.82.1:compile
...
Note it is clear from this that they have diverged and this will cause issues.
In this scenario we should NOT do the okhttp upgrade.
You’re maybe asking at this stage, why GRPC does not follow SemVer as 1.70 should be compatible to 1.82… but it is just public API calls that are compatible with minor releases, but SemVer splits the public API and the transport layer so calls to io.grpc:grpc-core are internal, according to their README and thus won’t be compatible. GRPC patch updates should be compatible.
Example: protobuf-java
Using mvn dependency:tree -Dincludes=com.google.protobuf we see
...
[INFO] com.demo:package:jar:1.0.0
[INFO] +- com.google.cloud:google-cloud-spanner:jar:6.93.0:compile
[INFO] | \- com.google.protobuf:protobuf-java-util:jar:3.25.5:compile
[INFO] \- com.google.protobuf:protobuf-java:jar:3.25.9:compile
...
Here we can see the actual project we use brings in 3.25.9 and com.google.cloud:google-cloud-spanner brings in 3.25.5.
In this case, as it is a patch version update, it should be safe to do the update the direct import to 3.25.9
Preventing issues
Finding issues at runtime is not ideal, it would be better to find them at compile time or using tests, and there are a few options for that.
Enforcer
Maven offers a plugin called Enforcer which provides goals to control certain environmental constraints, such as Maven version or the JDK version. For example, in this config it will ensure you use Maven 3.9
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<version>3.6.2</version>
<executions>
<execution>
<id>enforce-maven</id>
<goals>
<goal>enforce</goal>
</goals>
<configuration>
<rules>
<requireMavenVersion>
<version>3.9</version>
</requireMavenVersion>
</rules>
</configuration>
</execution>
</executions>
</plugin>
It offers two options directly for helping prevent issues Dependency Convergence and Ban Duplicate Classes.
Dependency Convergence
This feature checks that all transitive dependencies match and will fail the verify step if there is a change which causes a problem. If we made the GRPC version change from above, it will error as follows
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.6.3:enforce (enforce-runtime-safety) on project package:
[ERROR] Rule 1: org.apache.maven.enforcer.rules.dependency.DependencyConvergence failed with message:
[ERROR] Failed while enforcing releasability.
[ERROR]
[ERROR] Dependency convergence error for io.grpc:grpc-util:jar:1.70.0. Paths to dependency are:
[ERROR] +-com.demo:package:jar:1.0.0
[ERROR] +-com.google.cloud:google-cloud-spanner:jar:6.93.0:compile
[ERROR] +-io.grpc:grpc-util:jar:1.70.0:runtime
[ERROR] and
[ERROR] +-com.demo:package:jar:1.0.0
[ERROR] +-io.grpc:grpc-okhttp:jar:1.82.1:compile
[ERROR] +-io.grpc:grpc-util:jar:1.82.1:runtime
[ERROR]
[ERROR]
[ERROR] Dependency convergence error for io.grpc:grpc-core:jar:1.70.0. Paths to dependency are:
[ERROR] +-com.demo:package:jar:1.0.0
[ERROR] +-com.google.cloud:google-cloud-spanner:jar:6.93.0:compile
[ERROR] +-io.grpc:grpc-core:jar:1.70.0:compile
[ERROR] and
[ERROR] +-com.demo:package:jar:1.0.0
[ERROR] +-io.grpc:grpc-okhttp:jar:1.82.1:compile
[ERROR] +-io.grpc:grpc-core:jar:1.82.1:runtime
[ERROR]
[ERROR]
[ERROR] Dependency convergence error for io.grpc:grpc-api:jar:1.70.0. Paths to dependency are:
[ERROR] +-com.demo:package:jar:1.0.0
[ERROR] +-com.google.cloud:google-cloud-spanner:jar:6.93.0:compile
[ERROR] +-io.grpc:grpc-api:jar:1.70.0:compile
[ERROR] and
[ERROR] +-com.demo:package:jar:1.0.0
[ERROR] +-io.grpc:grpc-okhttp:jar:1.82.1:compile
[ERROR] +-io.grpc:grpc-api:jar:1.82.1:compile
In the error you can easily see the issues and which package is bringing in one version and not another, raising the error concern immediately. This goes even further than just obvious issues, it finds less than obvious issues too. For example here we have it finding an issue with com.google.guava:guava and it found 3 where are in use
[ERROR] Dependency convergence error for com.google.guava:guava:jar:33.4.0-jre. Paths to dependency are:
[ERROR] +-com.demo:package:jar:1.0.0
[ERROR] +-com.google.cloud:google-cloud-spanner:jar:6.93.0:compile
[ERROR] +-com.google.guava:guava:jar:33.4.0-jre:compile
[ERROR] and
[ERROR] +-com.demo:package:jar:1.0.0
[ERROR] +-io.grpc:grpc-okhttp:jar:1.70.0:compile
[ERROR] +-com.google.guava:guava:jar:33.3.1-android:runtime
[ERROR] and
[ERROR] +-com.demo:package:jar:1.0.0
[ERROR] +-io.confluent:kafka-protobuf-serializer:jar:7.9.8:compile
[ERROR] +-io.confluent:kafka-protobuf-provider:jar:7.9.8:compile
[ERROR] +-com.squareup.wire:wire-schema-jvm:jar:5.1.0:compile
[ERROR] +-com.google.guava:guava:jar:32.0.1-jre:compile
[ERROR] and
[ERROR] +-com.demo:package:jar:1.0.0
[ERROR] +-io.confluent:kafka-protobuf-serializer:jar:7.9.8:compile
[ERROR] +-io.confluent:kafka-schema-registry-client:jar:7.9.8:compile
[ERROR] +-com.google.guava:guava:jar:32.0.1-jre:compile
This is tricky to find as the depdency tree command mvn dependency:tree -Dincludes=com.google.guava only returns a single value, namely com.google.guava:guava:jar:33.4.0-jre:compile
[INFO] --- dependency:3.8.1:tree (default-cli) @ package ---
[INFO] com.demo:package:jar:1.0.0
[INFO] \- com.google.cloud:google-cloud-spanner:jar:6.93.0:compile
[INFO] +- com.google.guava:failureaccess:jar:1.0.2:compile
[INFO] +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile
[INFO] \- com.google.guava:guava:jar:33.4.0-jre:compile
What is happening in this scenario is Maven Nearest Wins strategy picks a single version to use, as described above, and in this case it picked com.google.guava:guava:jar:33.4.0-jre:compile because com.google.cloud:google-cloud-spanner was first in the pom.xml. If io.grpc:grpc-okhttp was first it would have picked com.google.guava:guava:jar:33.3.1-android:runtime and that would’ve caused crashes.
This letting Maven pick based on arbitrary ordering is not ideal, so you can specify the version you want using dependencyManagement in the pom.xml to specify exactly what you want. In this case we would pick the same as Maven did, but we guarantee that no matter what happens in the ordering of the pom.xml it just works, for example:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>com.google.guava:guava:jar:33.4.0-jre:compile</version>]
</dependency>
Why overriding -android with -jre is usually safe:
Guava is specifically designed so that the -jre flavour is a superset of the -android flavour. This means grpc-okhttp (which is asking for 33.3.1-android) will almost always work flawlessly with 33.4.0-jre because all the Android-compatible methods it needs are also present in the JRE version.
Why did we use a major change for guava without issue?
Looking at the example for Guava above we selected 33 which is a major version change compared to 32 wanted by io.confluent:kafka-schema-registry-client and, if it followed SemVer, that may indicate a potential risk, but Guava has additional notices
APIs without @Beta will remain binary-compatible for the indefinite future. (Previously, we sometimes removed such APIs after a deprecation period. The last release to remove non-@Beta APIs was Guava 21.0.) Even @Deprecated APIs will remain (again, unless they are @Beta). We have no plans to start removing things again, but officially, we’re leaving our options open in case of surprises (like, say, a serious security problem).
So this means it should be very stable.
Ban Duplicate Classes
The second feature of Enforcer looks at the generated classes and will see if multiple things have generated classes which conflict - these are impossible to find normally with the maven dependency tree.
For example io.swagger.core.v3:swagger-annotations:jar:2.1.10 and io.swagger.core.v3:swagger-annotations-jakarta:jar:2.2.47 end up creating the same generated code:
[ERROR] Rule 2: org.codehaus.mojo.extraenforcer.dependencies.BanDuplicateClasses failed with message:
[ERROR] Duplicate classes found:
[ERROR]
[ERROR] Found in:
[ERROR] io.swagger.core.v3:swagger-annotations:jar:2.1.10:compile
[ERROR] io.swagger.core.v3:swagger-annotations-jakarta:jar:2.2.47:compile
[ERROR] Duplicate classes:
[ERROR] io/swagger/v3/oas/annotations/media/Content.class
[ERROR] io/swagger/v3/oas/annotations/Parameter.class
[ERROR] io/swagger/v3/oas/annotations/media/ArraySchema.class
[ERROR] io/swagger/v3/oas/annotations/security/SecurityRequirement.class
[ERROR] io/swagger/v3/oas/annotations/info/License.class
[ERROR] io/swagger/v3/oas/annotations/responses/ApiResponse.class
[ERROR] io/swagger/v3/oas/annotations/media/Schema$AccessMode.class
[ERROR] io/swagger/v3/oas/annotations/media/Schema.class
[ERROR] io/swagger/v3/oas/annotations/parameters/RequestBody.class
[ERROR] io/swagger/v3/oas/annotations/headers/Header.class
[ERROR] io/swagger/v3/oas/annotations/enums/SecuritySchemeType.class
[ERROR] io/swagger/v3/oas/annotations/info/Info.class
[ERROR] io/swagger/v3/oas/annotations/media/DiscriminatorMapping.class
This is even worse in predicting what will be loaded, as it is not Maven but the JVM which decides, and it seems to be arbitrarily. It could randomly pick 2.1.10 and cause the app to crash at runtime. To fix scenarios like this, using dependency tree find what is bringing in the old version, for example:
mvn dependency:tree -Dincludes=io.swagger.core.v3:swagger-annotations
...
[INFO] --- dependency:3.8.1:tree (default-cli) @ project ---
[INFO] com.demo:package:jar:1.0.0
[INFO] \- io.confluent:kafka-protobuf-serializer:jar:7.9.8:compile
[INFO] \- io.confluent:kafka-schema-registry-client:jar:7.9.8:compile
[INFO] \- io.swagger.core.v3:swagger-annotations:jar:2.1.10:compile
[INFO] ------------------------------------------------------------------------
This points to io.confluent:kafka-protobuf-serializer:jar:7.9.8 so we will add an exclusion to the dependency to prevent it bringing in the old version
<dependency>
<groupId>io.confluent</groupId>
<artifactId>kafka-protobuf-serializer</artifactId>
<version>7.9.8</version>
<exclusions>
<exclusion>
<groupId>io.swagger.core.v3</groupId>
<artifactId>swagger-annotations</artifactId>
</exclusion>
</exclusions>
</dependency>
Spring Context Loading Test
What can be useful for Spring based projects is to have a test which can validate anything Spring uses can be loaded correctly:
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)class ApplicationRuntimeTest {
@Testvoid contextLoads() {
// If there is a catastrophic Guava or Protobuf mismatch,// the Spring context will crash right here during startup.
}
}
Note: that some scenarios will still not be found, for example if something is not loaded via Spring and in that case it would only fail when run.