Detecting Compatibility Issues With the Java SE API

2019 was a very busy year in terms of conferences. Our team could leave for whole weeks on business trips. As you know, the conference is a perfect time to share knowledge. In addition to giving talks and telling many interesting things at our booth, we also learned a lot from communicating with conference participants and speakers. So, at the Joker 2019 conference in the Fall, a talk from Dalia Abo Sheasha "Migrating beyond Java 8" inspired us to implement a new diagnostic rule that allows us to detect incompatibilities in the Java SE API between different versions of Java. This is what we will talk about.

Relevance of Detecting Java SE Compatibility Issues

At the moment, Java SE 14 has already been released. Despite this, many companies continue using previous versions of Java (Java SE 6, 7, 8,...). As time goes by and Java is constantly being updated, the problem of compatibility of different Java SE API versions becomes more and more relevant every year.

When new versions of Java SE are released, they are usually backward compatible with earlier versions, i.e., for example, an application developed on the basis of Java SE 8 should run without problems on the 11th Java version. However, in practice, some incompatibilities may occur in a number of classes and methods. This incompatibility is due to the fact that some APIs undergo changes: they are deleted, their behavior changes, they are marked as outdated, and much more.

This problem will only get worse when you start thinking about migrating your project to a newer Java SE version. Or when the technical support of your app will receive more and more emails saying that the app behaves incorrectly or can't start at all.

I think this is quite enough to get your attention here!

Existing Tools

There is no one-size-fits-all solution for migrating from one version of Java to another. Besides, if your application is a result of continuous multi-year development using a variety of non-trivial solutions, then when you bring yourself to upgrading to a fresh Java version, you are likely to engage in a rather time-consuming process.

This process involves identifying a problematic or potentially a problematic API that needs to be reviewed and replaced with an alternative solution. This can seriously affect the business logic, which can take a lot of time to fix and test. In case if not only do you need to upgrade to a new Java SE version, but also ensure compatibility with a number of versions, this task will become much more complicated.

After digging around the Internet on the topic of detecting API incompatibilities between different Java SE, I only met tools that come with JDK: javac, jdeps, jdeprscan.

There was no third-party tool for this case, except for the one that I had the honor to learn about listening to a Joker 2019 talk - Migration Toolkit for Application Binaries.

javac

When developing an application, one shouldn't forget about compiler warnings. Everyone has a different attitude to warnings:

• Someone is strictly watching them and sounds the alarm at every trifle. It might seem a bit paranoid, but it may well reduce technical risks in the future.
• Someone collects them in a separate file with the idea that they will review them as soon as they have time.
• Someone ignores the warnings at all.

The first option is decent since some problems can be solved without delay, for example, just don't use a method or class that is marked as outdated. Using this API isn't a blocking problem, but you should pay attention to it, since it becomes possible that when using a newly released Java version, your app will behave differently or even crash.

jdeps

If you are already concerned about migrating your application to a newer version of Java SE, then jdeps is here to help you.

jdeps is a command-line tool that performs static analysis of your application's dependencies and libraries by accepting *.class files or *.jar as input. Starting with Java SE 8, it comes bundled with the JDK.

What interests us here is that if you run this tool with the --jdk-internals option, it will tell you which internal JDK API each of your classes depends on. This is a very important point because the internal API doesn't guarantee that it won't change in future versions of Java.

Let's look at an example. Let's say you have been developing your application on Java 8 for a long time and have never been questioned by the compatibility of the Java SE API used with more recent versions. Here comes the question about switching your application to, for example, Java 11. In this case, you can take the path of least resistance and immediately launch the application using Java 11. But something might go wrong, and let's say your app launch ended up crashing. In this case, you can run jdeps from Java 11 by feeding it your application's *.jar files.

The result is about to be as follows:

The output shows that the sun.misc.BASE64Encoder dependency in Java 11 will be removed. And the crash of your application when first running on Java 11 is most likely due to the java.lang.NoClassDefFoundError error. In addition to this information, which can't help but please us, jdeps can offer you an alternative dependency that can be used instead of the current one. In this case, it suggested replacing the removed dependency with java.util.Base64.

The second warning of the tool indicates that we are still using another internal dependency in the code, but it is still correct. We don't know if there will be changes in this API in the future Java versions, but we should take notice of it.

Of course, jdeps can do more than just test the use of internal JDK components. This isn't part of the scope of this article, but you can find its features on the official page yourself.

jdeprscan

The goals of jdeprscan are exactly the same as those of jdeps, namely, help in finding an unwanted and problematic API.

jdeprescan is a static analysis tool that scans a *.jar file (or some other set of *.class files) for outdated API elements. Using an outdated API isn't a blocking problem, but one should pay attention to it. Starting with Java SE 9, it comes bundled with the JDK.

Let's also assume that there is an issue of migrating the application to Java 11. In this case, run the command

jdeprscan --release 8 app.jar

and you will get a list of APIs that are no longer recommended for Java 8, i.e. the API that may be removed in future Java versions. After fixing all the warnings, you can run

xxxxxxxxxx

jdeprscan --release 11 app.jar

which will output a list of APIs that are already deprecated for Java 11. This way, you can find and fix (if necessary) the entire non-recommended API.

Migration Toolkit for Application Binaries

This tool is designed to help you quickly evaluate your user application for potential problems before deploying on various servers (JBOSS, WebShere, Tomcat, WebLogic, ...). In addition to all the functionality, the tool also allows you to detect differences in the Java SE API of different versions.

Let's take a quick look at what this tool is.

Quick launch of the tool looks like this:

xxxxxxxxxx

java -jar binaryAppScanner.jar yourApp.jar --analyzeJavaSE 

--sourceJava=oracle8 --targetJava=java11 ....

The analyzeJavaSE option uses various parameters, which you can find out about by calling help.

After running the analysis, you will soon see a report in your web browser:

The screen doesn't fit it entirely =(. And while you haven't yet tried to run this tool, I will describe it in words.

The report shows you rules with 3 severity levels:

• High-severity rules (deleted APIs, behavior changes that make the app unusable and require correction).
• Warning rules (changes to API behavior that may cause problems and are worth investigating).
• Information rules (use of outdated APIs, API behavior that may slightly change behavior, but doesn't affect the program).

Each warning can be expanded to see a description with all the details. Information messages contain a recommendation for you to run jdeps in addition to current ones. They say they are focused on migrating the application, and jdeps will additionally help to detect the problem in internal JDK packages (in addition to what they find).

You can also find a list of rules that were used for the analysis at the bottom of the report.

If you are using the Eclipse IDE, you can use the plugin. For a more detailed study, you are welcome to visit their page.

Implementation in PVS-Studio

After studying the tools discussed, we came to the conclusion that finding potential compatibility problems with different versions of the Java SE API is a worthy task for static analysis.

These tools will really make it easier to migrate your app or find an API that might break your app's performance on newer versions of Java SE (if you don't want to upgrade your app). However, after thinking that these tools always need to be run on the command line separately from the development process to identify problems, we came to the decision that this isn't very convenient. 

Based on the fact that static analysis is necessary for detecting problematic or potentially problematic code at the earliest stages of development, we implemented the V6078 diagnostic rule, which will signal you about the "problematic" API.

The V6078 rule will warn you in advance that your code is dependent on certain functions and classes of the Java SE API, which may cause you difficulties in future versions of Java. Besides, at the very beginning of implementing a particular feature you won't get tied to this API, thereby reducing technical risks in the future.

The diagnostic rule issues warnings in the following cases:

• If the method/class/package is deleted in the target Java version;
• If the method/class/package is marked as deprecated in the target Java version;
• If the method's signature has changed.

The rule currently allows you to analyze the compatibility of Oracle Java SE from versions 8 to 14. To make the rule active, you must configure it.

IntelliJ IDEA

In the IntelliJ IDEA plugin, you need to enable the rule in the tab Settings > PVS-Studio > API Compatibility Issue Detection and specify the parameters, namely:

• Source Java SE - the Java version that your application is developed on.
• Target Java SE - the Java version, which you want to check the compatibility with, of the API used in your application (Source Java SE).
• Exclude packages - packages that you want to exclude from compatibility analysis (packages are separated by commas).

Plugin for Gradle

Using the gradle plugin, you need to configure the analyzer settings in build.gradle:

xxxxxxxxxx

apply plugin: com.pvsstudio.PvsStudioGradlePlugin

pvsstudio {

    ....

    compatibility = true

    sourceJava = /*version*/  

    targetJava = /*version*/

    excludePackages = [/*pack1, pack2, ...*/]

}

Plugin for Maven

Using the maven plugin, you have to configure the analyzer settings in pom.xml:

xxxxxxxxxx

<build>

  <plugins>

    <plugin>

      <groupId>com.pvsstudio</groupId>

      <artifactId>pvsstudio-maven-plugin</artifactId>

      ....

      <configuration>

        <analyzer>

          ....

          <compatibility>true</compatibility>

          <sourceJava>/*version*/</sourceJava>

          <targetJava>/*version*/</targetJava>

          <excludePackages>/*pack1, pack2, ...*/</excludePackages>        

        </analyzer>

      </configuration>

    </plugin>

  </plugins>

</build>

Using the Core Directly

If you use the analyzer directly from the command line, you have to use the following parameters to enable compatibility analysis of the selected Java SE API:

xxxxxxxxxx

java -jar pvs-studio.jar /*other options*/ --compatibility 

--source-java /*version*/ --target-java /*version*/ 

--exclude-packages /*pack1 pack2 ... */

Analyzer Triggering

Let's assume that we are developing an application based on Java SE 8, and we have a class with the following content:

xxxxxxxxxx

/* imports */

import java.util.jar.Pack200;

public class SomeClass

{

  /* code */

  public static void someFunction(Pack200.Packer packer, ...)

  {

    /* code */

    packer.addPropertyChangeListener(evt -> {/* code */});

    /* code */

  }

}

By running static analysis with different settings for the diagnostic rule, we will see the following picture:

• Source Java SE - 8, Target Java SE - 9
  • The 'addPropertyChangeListener' method will be removed.
• Source Java SE - 8, Target Java SE - 11
  • The 'addPropertyChangeListener' method will be removed.
  • The 'Pack200' class will be marked as deprecated.
• Source Java SE - 8, Target Java SE - 14
  • The 'Pack200' class will be removed.

First, the addPropertyChangeListener method in the Pack200.Packer class was removed in Java SE 9. In version 11, this was supplemented by the fact that the 'Pack200' class was marked as deprecated. In version 14, this class was removed at all.

Therefore, when running the application on Java 11, you will get java.lang.NoSuchMethodError, and if run on Java 14 - java.lang.NoClassDefFoundError.

Knowing this information, when developing your app, you will consider alternative solutions to the task at hand.

Ideas for Further Development

During the implementation of the diagnostic rule there were ideas for expanding:

• Since jdeprscan and jdeps can't warn you about using reflection to access the encapsulated API, it makes sense to modify the rule so that it tries to find out which API is being used. The result may not be perfect, but why not?!
• There is a wide variety of JDK implementations (from Oracle, IBM, Red Hat,...). As a rule, they are compatible with each other. To what extent does the internal JDK API differ considerably? After all, developers can get hooked on it, which can lead to potential problems when switching from one JDK to another.

These are all research questions at this point. Time will show what will happen in the end =) If you know interesting scenarios for this diagnostic operation and would like to see them in PVS-Studio, then write to us.

Conclusion

Searching for potential incompatibility errors is fully consistent with our ideology - using PVS-Studio to find and fix errors at the early stages of code writing. Same as a typo, you can add a specific function call to the code at any time, so it is now twice as useful to run PVS-Studio regularly on a project.

The V6078 diagnostic is available in the analyzer starting from version 7.08. You can download and try the analyzer on your project on the download page.