New RegEx Features in Java 9

I recently received my complimentary copy of the book “Java 9 Regular Expressions” from Anubhava Srivastava published by Packt. The book is a good tutorial and introduction to anyone who wants to learn what regular expressions are and start from scratch. For those who know how to use RegEx, the book may still be interesting to reiterate the knowledge and to deepen into more complex features like zero length assertions, back references, and the like.

In this article, I will focus on the few regular expression features that are specific to Java 9 and were not available in earlier version of the JDK.

Java 9 Regular Expression Module

The JDK in Java 9 is split up into modules. One could rightfully expect that there is a new module for regular expression handling packages and classes. Actually, there is none. The module java.base is the default module on which all other modules depend on by default, and thus the classes of the exported packages are always available in Java applications. The regular expression package java.util.regex is exported by this module. This makes the development a bit simpler: There is no need to explicitly ‘require’ a module if we want to use regular expressions in our code. It seems that regular expressions are so essential to Java that it got included in the base module.

Regular Expression Classes

The package java.util.regex contains the classes:

• MatchResult
• Matcher
• Pattern
• PatternSyntaxException

The only class that has changed APIs is Matcher.

Matcher Class Changes

The class Matcher adds five new methods. Four of those are overloaded versions of already existing methods. These are:

• appendReplacement
• appendTail
• replaceAll
• replaceFirst
• results

The first four exist in earlier versions and there is only change in the types of the arguments (after all, that is what overloading means).

appendReplacement/Tail

In the cases of appendReplacement and appendTail , the only difference is that the argument can also be a StringBuilder and not only a StringBuffer. Considering that StringBuilder was introduced in Java 1.5, something like 13 years ago, nobody should say that this is an inconsiderate act.

It is interesting, though, how the currently online version of the API JDK documents the behaviour of appendReplacement for StringBuilder arguments. The older StringBuffer-argumented method explicitly documents that the replacement string may contain named references that will be replaced by the corresponding group. The StringBuilder argumented version misses this. The documentation seems like it was copy/pasted and then edited. The text replaces “buffer” to “builder” and the like, and the text documenting the named reference feature is deleted.

I tried the functionality using Java 9 build 160 and the outcome is the same for these two method versions. This should not be a surprise, since the source code of the two methods is the same, a simple copy/paste in the JDK with the exception of the argument type.

Seems that you can use:

@Test
public void testAppendReplacement() {
 
    Pattern p = Pattern.compile("cat(?<plural>z?s?)");
    //Pattern p = Pattern.compile("cat(z?s?)");
    Matcher m = p.matcher("one catz two cats in the yard");
    StringBuilder sb = new StringBuilder();
    while (m.find()) {
        m.appendReplacement(sb, "dog${plural}");
        //m.appendReplacement(sb, "dog$001");
    }
    m.appendTail(sb);
    String result = sb.toString();
    assertEquals("one dogz two dogs in the yard", result);
}

Both the commented lines or the line above each. The documentation, however, speaks only about the numbered references.

replaceAll/First

This is also an “old” method that replaces matched groups with some new strings. The only difference between the old version and the new is how the replacement string is provided. In the old version, the string was given as a String calculated before the method was invoked. In the new version, the string is provided as a Function<MatchResult,String>. This function is invoked for each match result, and the replacement string can be calculated on the fly.

Knowing that the class Function was introduced only three years ago in Java 8, the new use of it in regular expressions may be a little slap-dash. Or, perhaps… may be we should see this as a hint that, 10 years from now, when the class Fuction will be 13 years old, we will still have Java 9?

Let's dig a bit deeper into these two methods. (Actually only to replaceAll because replaceFirst is the same except that it replaces only the first matched group.) I tried to create some not-absolutely-intricate examples where such a use could be valuable.

The first sample is from the JDK documentation:

@Test
public void demoReplaceAllFunction() {
    Pattern pattern = Pattern.compile("dog");
    Matcher matcher = pattern.matcher("zzzdogzzzdogzzz");
    String result = matcher.replaceAll(mr -> mr.group().toUpperCase());
    assertEquals("zzzDOGzzzDOGzzz", result);
}

It is not too complex and shows the functionality. The use of a lambda expression is absolutely adequate. I cannot imagine a simpler way to uppercase the constant string literal “dog”. Perhaps only writing “DOG”. Okay, I am just kidding. But really, this example is too simple. It is okay for the documentation where anything more complex would distract the reader from the functionality of the documented method. Really: Do not expect less intricate examples in a Javadoc. It describes how to use the API and not why the API was created and designed that way.

But here and now, we will look at some more complex examples. We want to replace the # characters in a string with the numbers 1, 2, 3, and so on. The string contains numbered items and, if we insert a new one into the string, we do not want to renumber it manually. Sometimes, we group two items, in which case we write ## and then we just want to skip a serial number for the next #. Since we have a unit test, the code describes the functionality better than I can put it into words:

@Test
public void countSampleReplaceAllFunction() {
    AtomicInteger counter = new AtomicInteger(0);
    Pattern pattern = Pattern.compile("#+");
    Matcher matcher = pattern.matcher("# first item\n" +
            "# second item\n" +
            "## third and fourth\n" +
            "## item 5 and 6\n" +
            "# item 7");
    String result = matcher.replaceAll(mr -> "" + counter.addAndGet(mr.group().length()));
    assertEquals("1 first item\n" +
            "2 second item\n" +
            "4 third and fourth\n" +
            "6 item 5 and 6\n" +
            "7 item 7", result);
}

The lambda expression passed to replaceAll gets the counter and calculates the next value. If we used one #, then it increases it by 1. If we used two, then it adds two to the counter, and so on. Because a lambda expression cannot change the value of a variable in the surrounding environment (the variable has to be effectively final), the counter cannot be an int or Integer variable. We need an object that holds an int value and can be changed. AtomicInteger is exactly that, even if we do not use the atomic feature of it.

The next example goes even further and does some mathematical calculation. It replaces any floating point formatted number in the string to the sine value of it. That way, it corrects our sentence, since sin(pi) is not even close to pi, which cannot be precisely expressed here. It is rather close to zero:

@Test
public void calculateSampleReplaceAllFunction() {
    Pattern pattern = Pattern.compile("\\d+(?:\\.\\d+)?(?:[Ee][+-]?\\d{1,2})?");
    Matcher matcher = pattern.matcher("The sin(pi) is 3.1415926");
    String result = matcher.replaceAll(mr -> "" + (Math.sin(Double.parseDouble(mr.group()))));
    assertEquals("The sin(pi) is 5.3589793170057245E-8", result);
}

We will also play around a bit with this calculation for the demonstration of the last method in our list, which is a brand new one in the Matcher class.

Stream results()

The new method results() returns a stream of the matching results. To be more precise, it returns a Stream of MatchResult objects. In the example below, we use it to collect any floating point formatted number from the string and print their sine value, comma separated:

@Test
public void resultsTest() {
    Pattern pattern = Pattern.compile("\\d+(?:\\.\\d+)?(?:[Ee][+-]?\\d{1,2})?");
    Matcher matcher = pattern.matcher("Pi is around 3.1415926 and not 3.2 even in Indiana");
    String result = String.join(",",
            matcher
                    .results()
                    .map(mr -> "" + (Math.sin(Double.parseDouble(mr.group()))))
                    .collect(Collectors.toList()));
    assertEquals("5.3589793170057245E-8,-0.058374143427580086", result);
}

The new regular expression methods introduced in the Java 9 JDK are not essentially different from what was already available. They are neat and handy and, in some situation, they may ease programming. There is nothing that could not have been introduced in an earlier version, though. This is just the way of Java — to make such changes to the JDK slow and well-thought out. After all, that is why we love Java, don’t we?

The whole code, copy/pasted from my IDE, can be found and downloaded from the following gist.