[LinkSet] Dependency duplicates in Maven and Jar Hell with Java Classloader


Theoretical part:
Java Classloader

Maven: Introduction to the Dependency Mechanism

What is JAR hell? (Or is it classpath hell? Or dependency hell?)

Jar Hell made Easy — Demystifying the classpath with jHades
See JHades documentation, it very useful to find overlapping jars.

Another tool for dealing with Jar Hell is Weblogic Classloader Analysis Tool.

One of the most problematic dependency is Xeres Xerces Hell. It’s a good example how not to do library.

This presentation is a great resource about Jar Hell and the different type of classpath related exceptions:

But little bit boring.

Maven Dependency Wrangling

dealing with dependency chain issues in maven

Maven Enforcer plugin has extra rule
Ban Duplicate Classes
Also it should be very useful if you have legacy project that still runs under Java 6 or 7 you should avoid dependencies compiled with never Java 8.
You can use enforceBytecodeVersion

Please also make sure that you also specified requireJavaVersion (to compile), requireUpperBoundDeps, requireReleaseDeps, requirePluginVersions and other useful standard rules .

Also if you have a submodules in project it will be also useful ono-extra-enforcer-rules

So, your enforcer rules may looks like:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-enforcer-plugin</artifactId>
    <version>1.4.1</version>
    <executions>
        <execution>
            <id>enforce</id>
            <goals>
                <goal>enforce</goal>
            </goals>
            <configuration>
                <rules>
                    <bannedPlugins>
                        <!-- will only display a warning but does not fail the build. -->
                        <level>WARN</level>
                        <excludes>
                            <exclude>org.apache.maven.plugins:maven-verifier-plugin</exclude>
                        </excludes>
                        <message>Please consider using the maven-invoker-plugin (http://maven.apache.org/plugins/maven-invoker-plugin/)!</message>
                    </bannedPlugins>
                    <requireMavenVersion>
                        <version>3.0.5</version>
                    </requireMavenVersion>
                    <requireJavaVersion>
                        <version>1.8</version>
                    </requireJavaVersion>
                    <requireReleaseDeps>
                        <onlyWhenRelease>true</onlyWhenRelease>
                        <message>No Snapshots Allowed!</message>
                    </requireReleaseDeps>
                    <requireUpperBoundDeps>
                        <!-- 'uniqueVersions' (default:false) can be set to true if you want to compare the timestamped SNAPSHOTs  -->
                        <!-- <uniqueVersions>true</uniqueVersions> -->
                    </requireUpperBoundDeps>
                    <reactorModuleConvergence>
                        <message>The reactor is not valid</message>
                        <ignoreModuleDependencies>true</ignoreModuleDependencies>
                    </reactorModuleConvergence>
                    <requirePluginVersions>
                        <message>Best Practice is to always define plugin versions!</message>
                        <banLatest>true</banLatest>
                        <banRelease>true</banRelease>
                        <banSnapshots>true</banSnapshots>
                        <phases>clean,deploy,site</phases>
                        <additionalPlugins>
                            <additionalPlugin>org.apache.maven.plugins:maven-eclipse-plugin</additionalPlugin>
                            <additionalPlugin>org.apache.maven.plugins:maven-reactor-plugin</additionalPlugin>
                        </additionalPlugins>
                        <unCheckedPluginList>org.apache.maven.plugins:maven-enforcer-plugin,org.apache.maven.plugins:maven-idea-plugin</unCheckedPluginList>
                    </requirePluginVersions>
                    <enforceBytecodeVersion>
                        <maxJdkVersion>1.6</maxJdkVersion>
                        <excludes>
                            <exclude>org.mindrot:jbcrypt</exclude>
                        </excludes>
                    </enforceBytecodeVersion>
                    <banDuplicateClasses>
                        <ignoreClasses>
                            <!-- example of ignoring one specific class -->
                            <ignoreClass>com.xyz.i18n.Messages</ignoreClass>

                            <!-- example of ignoring with wildcards -->
                            <ignoreClass>org.apache.commons.logging.*</ignoreClass>
                        </ignoreClasses>
                        <findAllDuplicates>true</findAllDuplicates>
                    </banDuplicateClasses>
                    <banCircularDependencies/>
                    <ForbidOverridingManagedDependenciesRule>
                        <excludes>
                            <!-- guava in parent is too old, so allow to override it -->
                            <exclude>com.google.guava:guava</exclude>
                        </excludes>
                    </ForbidOverridingManagedDependenciesRule>
                    <ForbidOverridingManagedPluginsRule/>
                    <ForbidDependencyManagementInSubModulesRule/>
                    <ManageAllModulesRule/>
                </rules>
            </configuration>
        </execution>
    </executions>
    <dependencies>
        <dependency>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>extra-enforcer-rules</artifactId>
            <version>1.0-beta-3</version>
        </dependency>
        <dependency>
            <groupId>net.oneandone.maven</groupId>
            <artifactId>ono-extra-enforcer-rules</artifactId>
            <version>0.1.1</version>
        </dependency>
    </dependencies>
</plugin>

Two attempts to find duplicated classes with Maven
Remove duplicate classes the agile way: Maven Duplicate Finder Plugin

Finding Duplicate Class Definitions Using Maven

Both of this plugins are discussed here:
Figuring out duplicate class definitions using the Analyze goal

Also maven-shade-plugin does check for overlapping classes during packaging of uber-jar.

Resolving conflicts using the dependency:tree -Dverbose
It shows which dependencies are duplicated (omitted for duplicate), with are evicted with newer version (version managed from 1.6) but it doesn’t show which dependencies was excluded.

Another one good thing that worst to do is enable Failing the build on dependency analysis warnings. Note: it binded to verify phase that runed after package.

JDEPS Java Dependency Analysis Tool from JDK 8

Also some related articles

Versions compitable

For example changelog of Joda-Time v2.9

Compatibility with 2.8

Build system — Yes
Binary compatible — Yes
Source compatible — Yes
Serialization compatible — Yes
Data compatible — Yes
— DateTimeZone data updated to version 2015g
Semantic compatible — Yes

See another [Linkset] Compatibility

Jigsaw

It is possible a situation when some two libraries wanting to use the same dependency but of different versions. Unfortunately, in this cases we can’t manage this and should use -nodep versions.
Finally this problem will be resolved in JDK 9 Jigsaw: a jar can be declared as a module and it will run in it’s own isolated class loader, that reads class files from other similar module class loaders in an OSGI sort of way.
This will allow multiple versions of the same Jar to coexist in the same application if needed.

Working with deprecation

Upgrading of dependncies may require to remove some old codebase that depends on them.
This is also should be done in right way, so here some links that may helps:
* JEP 277
* Dr. Deprecator Prescriptions: important things that you should know about obsolete Java API

Speed up maven build

It’s also related topic. The main reason why I decided to add it here is because usually during speeding up build you will find a lot of problems with dependency graph.
It will helps you to make yoir project more modulized. Also for example paralell build may fails if your tests are in conflict (shares same resources, for example integration tests may use the same port).

Dependency analyzers

Also useful

* jApiCmp japicmp is a tool to compare two versions of a jar archive
* Java API Compliance Checker: A Perl script that uses javap to compare two jar archives. This approach cannot compare annotations and you need to have Perl installed.
* Clirr: A tool written in Java that compares two libraries for binary compatibility. Tracking of API changes is implemented only partially, tracking of annotations is not supported. Development has stopped around 2005.
* JDiff: A Javadoc doclet that generates an HTML report of all API changes. The source code for both versions has to be available, the differences are not distinguished between binary incompatible or not. Comparison of annotations is not supported.
* revapi: An API analysis and change tracking tool that was started about the same time as japicmp. It ships with a maven plugin and an Ant task, but the maven plugin currently (version 0.4.1) only reports changes on the command line.

Реклама

4 comments

  1. Уведомление: [Linkset] Migration to Java 8 | stokito on software
  2. Anastasiia Smirnova

    enforcer plugin шикарная вещь. жаль, только, что дерево зависимостей он строит все еще на основе 2го мейвена. А ведь в 3м добавили wildcard exclusion.. И вот незадача, не получится удобно им пользоваться если пользуешься эксклудом всех зависимостей в 3м мейвене 😦
    https://issues.apache.org/jira/browse/MENFORCER-232

    • stokito

      Спасибо за информацию. Мне только не понятно зачем экслкудить все депендеси?
      Я был бы очень благодарен если бы ты описала свой опыт и проблемы с которыми сталкивалась

  3. Уведомление: Dependencies Hell: Конфликт зависимостей и версионные выбоины | stokito on software

Добавить комментарий

Заполните поля или щелкните по значку, чтобы оставить свой комментарий:

Логотип WordPress.com

Для комментария используется ваша учётная запись WordPress.com. Выход / Изменить )

Фотография Twitter

Для комментария используется ваша учётная запись Twitter. Выход / Изменить )

Фотография Facebook

Для комментария используется ваша учётная запись Facebook. Выход / Изменить )

Google+ photo

Для комментария используется ваша учётная запись Google+. Выход / Изменить )

Connecting to %s