The Capability Pattern: Future-Proof Your APIs

Here is a simple pattern which you can use to make your APIs extensible, even by third parties, without sacrificing your ability to keep backward compatibility.

It is very frequent to create a library which has two “sides” — an API side and an SPI side. The API is what applications call to use the library. The SPI (Service Provider Interface) is how functionality — for example, access to different kinds of resources, is provided.

One example of this is JavaMail: To read/write email messages, you call JavaMail's API. Under the hood, when you ask for a mail store for, say, an IMAP mail server, the JavaMail library looks up all the providers registered (injected) on the classpath, and tries to find one that supports that protocol. The protocol handler is written to JavaMail's SPI. If it finds one, then you can fetch messages from IMAP servers using it. But your client code only ever calls the JavaMail API - it doesn't need to know anything about the IMAP service provider under the hood.

There is one very big problem with the way this is usually done: API classes really ought to be final in almost all cases. SPI classes ought to be abstract classes unless the problem domain is extremely well-defined, in which case interfaces make sense (you can use either, but in a not-well-defined problem domain you may end up, over time, creating things with awful names like LayoutManager2).

I won't go into great detail about why this is true here (my friend Jarda does in his new book and we discuss it somewhat in our book Rich Client Programming). In abbreviated form, the reasons are:

1. You can provably backward compatibly add methods to a final class. And if the class is final, that fact has communication-value — it communicates to the user of that class that it's not something they might need to implement, where an interface would be more confusing.


2. You can backward compatibly remove methods from an SPI interface or abstract class, if your library is the only thing that will ever call the SPI directly is your library. Older implementations will still have the method, it just will never be called (in a modular environment such as the NetBeans module system, OSGi or presumably JSR-277, you would enforce this by putting the API and SPI in separate JAR files, so a client can't even see the SPI classes).

A minor benefit of using abstract classes is that you can semi-compatibly add non-abstract methods to an abstract class later. But do remember that you run the risk that someone will have a subclass with the same method name and arguments and an incompatible return-type (the JDK actually did this to us once in NetBeans, by adding Exception.getCause() in JDK 1.3). So adding methods to a public, non-final class in an API is a backward-incompatible change.

Given those constraints, what happens if you mix API and SPI in the same class (which is what JavaMail and most Java standards do)? Well, you can't add methods compatibly because that could break subclasses. And you can't remove them compatibly, because clients could be calling them. You're stuck. You can't compatibly add or remove anything from the existing classes.

As I've written elsewhere, it is the height of insanity that an application server vendor is supposed to implement interfaces and classes that its clients directly call — for exactly this reason. It would be much cleaner, and allow Java APIs to evolve much faster, if API and SPI were completely separated.

But part of the appeal to vendors, for better or worse, to implement these specifications, is that they can extend them in custom ways that will tie developers who use those extensions to their particular implementation. This behavior not entirely about being evil and locking people in. There is a genuine case for innovation on top of a standard - that's how standards evolve, and some people will need functionality that the standard doesn't yet support.

Enter the capability pattern. The capability pattern is very, very simple. It looks like this:

public <T> getCapability (Class<T> type);
      

That's it! It's incredibly simple! It has one caveat: Any call to getCapability() must be followed by a null-check. But this is much cleaner than either catching UnsupportedOperationExceptions, or if (foo.isAbleToDoX()) foo.doX() or if (foo instanceof DoerOfX) ((DoerOfX) foo).doX(). A null-check is nice and simple and clean by comparison. It's letting the Java type system work for you instead of getting into a wrestling match with it.

Now, what can you do with it? Here's an example. In my previous blog I introduced an alternative design for how you could do something like SwingWorker. It contains a class called TaskStatus, which abstracts the task status data from the task-performing object itself. It is a simple interface with setters that allow a background thread to inform another object (presumably a UI) about the progress of a task.

In light of what we just discussed, TaskStatus really ought to be a final class. So let's rewrite it a little, to look like this. We will use a mirror-class for the SPI.

public final class TaskStatus {
    private final StatusImpl impl;
    TaskStatus (StatusImpl impl) {
        this.impl = impl;
    }

    public void setTitle (String title) {
        impl.setTitle (title);
    }

    public void setProgress (String msg, long progress, long min, long max) {
        //We could do argument sanity checks here and make life
        //simpler for anyone implementing StatusImpl
        impl.setProgress (msg, progress, min, max);
    }

    public void setProgress (String msg) {
        //...you get the idea
        //...
}

public abstract class StatusImpl {
    public abstract void setTitle (String title);
    public abstract void setProgress (String msg, long progress, long min, long max);
    public abstract void setProgress (String msg); //indeterminate mode
    public abstract void done();
    public abstract void failed (Exception e);
}
      

So we have an API that handles basic status display. But people are going to invent new aspects to status display. We can't save the world and solve everybody's task-status problems before they even think of them - and we shouldn't try. We don't want to set things up so that it's up to us to implement everything the world will ever want. Luckily, it doesn't have to be that way.

Since we've designed our API so that it can be compatibly added to, we let the rest of the world come up with things they need for displaying task status, and the ones that a lot of people need can be added to our API in the future. The capability pattern lets us do that. We add two methods to our API and SPI classes:

public abstract class StatusImpl {
    //...
    public <T> T getCapability (Class<T> type);
}
public final class TaskStatus {
    //...
    public <T> T getCapability (Class<T> type) {
        return impl.getCapability (type);
    }
}
    

Let's put that to practical use. Someone might want to display how much time remains before the task is done. Our API doesn't handle that. Through the capability pattern, we can add that. We (or anyone implementing StatusImpl) can create the following interface:

public interface StatusTime {
    public void setTimeRemaining (long milliseconds);
}
      

A task that wants to provide this information to the UI, if the UI supports it, simply does this:

public T runInBackground (TaskStatus status) {
   StatusTime time = status.getCapability (StatusTime.class);
   for (...) {
      //do some slow work...
      if (time != null) {
          long remaining = //estimate the time remaining
          time.setTimeRemaining (remaining);
      }
   }
}
      

Even better, our Task API is, right now, not tied specifically to Swing or AWT - it could be used for anything that needs to follow the pattern of computing something on a background thread and then doing work on another one. Why not keep it un-tied to UI toolkits? All we have to do is make the code that actually handles the threading pluggable (I'll talk about how you do this simply using the Java classpath for dependency injection in my next blog). Then the result could be used with SWT or Thinlet as well, or even in a server-side application. Instead of a SwingWorker, we have an AnythingWorker!

But we know we need a UI - and we know we are targetting Swing right now. How can we really keep this code completely un-tied from UI code and still have it be useful?

The capability pattern comes to our rescue again - very very simply. An actual application using this UI simply fetches the default factory for StatusImpls (you need such a thing if you want to run multiple simultaneous background tasks and show status for each — my next blog will explain how this can be injected just by putting a JAR on the classpath) and does something like:

Component statusUi = theFactory.getCapability (Component.class);
if (statusUi != null) {
    statusBar.add (statusUi);
}
      

(or if we want to allow only one background task at a time, we can forget the factory and put the Component fetching code directly in our implementation of StatusImpl).

If you are familiar with NetBeans Lookup API, the capability pattern is really a simplification of that (minus collection-based results and listening for changes).

The point here is that the capability pattern lets you have an API that is composed completely of nice, future-proofed, evolvable, final classes, but the API is extensible even though it is final. The result is that the API can evolve faster, with fewer worries about breaking anybody's existing code. Which reduces the cycle time to improve existing libraries, and all our software evolves and improves faster, which is good for everyone.

It also helps one to avoid trying to “save the world” — by allowing for extensibility, it is possible to create an API that is useful without needing to handle every possible thing anyone might ever want to do in that problem domain. Trying to save the world is what leads to scope-creep and never-finished projects. In this tutorial I discuss the don't try to save the world principle in a practical example.

Does the mirror-class design seem a bit masochistic? I think it does point up a weakness in the scoping rules of the Java language. It would definitely be nicer to be able to, on the method level, make some methods visible to some kinds of clients, and other methods visible to other kinds of clients. But regardless of this, it's even more masochistic to end up “painted into a corner,”^[1] and unable to fix bugs or add features without potentially breaking somebody's code. That's how you end up with ten-year-old unfixed bugs.

^[1]painted into a corner — An English idiom meaning to leave yourself with no options — you were painting the floor of a room in a pattern such that you end up standing in an unpainted corner of the room, and you can't leave the corner until the paint dries.

From http://weblogs.java.net/blog/timboudreau/