Debugging the Perplexing - Don’t Panic!

In the 10+ years, I’ve spent in software development, I’ve formulated a law of debugging: “The perplexity of a software bug and the simplicity of its probable cause are positively correlated”. Put simply, the more confounding and “impossible” a bug appears to be, the likelier it is that the underlying reason for the bug is not some nightmare compiler edge-case or hardware problem, but rather something that’s actually quite simple. Below are two cases that demonstrate the law in action.

Case #1: Python

My go-to example for this used to be an error in a Python project that took almost an entire day to resolve. Below is a (highly simplified) representation of the code in question:

def print_hello():
    line = "Hello"
    print(line)

Given the entire three lines of code involved, you can imagine that the executing program consistently prints out the error message NameError: name ‘line’ is not defined on the line of code print(line) drove me to start questioning my sanity. After all, the variable line was defined on the line *right above* where it was being used; how could the Python interpreter then turn around and claim that the variable didn’t exist? Eventually, I came across the reason for the error in one of the most happenstance of ways: I opened up the file in a text editor that displayed whitespace characters by default. Suddenly, the issue became as clear as day:

def print_hello():
    line = "Hello"
····print(line)

As in, line 2 was indented with a tab character, whereas line 3 was indented with four spaces. That sound you hear is every Python veteran cringing!

For those less familiar with Python, a quick explainer: Python is a scripting language that’s renowned for eschewing curly braces, parentheses, keywords, etc. in favour of spacing for defining code blocks. While this means that Python code may look “cleaner” than its equivalent in languages like Java, Ruby, or C++, it also means that the code’s indentation is no longer merely a style suggestion – it now determines how the code is executed. In this case, the Python evaluator determined that the tab-indented line = “Hello” was in its own scope compared to the space-indented print(line), thus it created the line variable in line 2 for the tab-indented scope; exited out of that scope when the line had finished executing; and proceeded to raise the NameError on line 3 because there was now no variable named line available! As aggravating as it was to see such a simple error lying in front of me, it also meant that the error was not Python breaking and instead one that had the simple fix of replacing the tab character with four spaces. Thankfully, the Python world has very much improved since this incident: modern IDEs will identify the spacing discrepancy, and Python 3 will raise either a TabError or an IndentationError (depending on the order of the tab- and space-indented lines in the code) instead of a mystifying error like the above.

Case #2: Lombok

(Special thanks to David Garcia Folch for helping to discover and analyze this issue!)

Those who have written Java code will likely have worked with Lombok on at least one project during their career. For the rest: Lombok is a library for Java that contains a set of annotations used for generating boilerplate Java code. To give an example, the following code:

import lombok.AllArgsConstructor;
import lombok.Data;

@AllArgsConstructor
@Data
public class SomeClass {
   private String foo;

   public void doSomething() {
       System.out.println("Doing something");
   }
}

Proceeds to generate the following (compiled) code:

public class SomeClass {
   private String foo;

   public void doSomething() {
       System.out.println("Doing something");
   }

   public SomeClass(String foo) {
       this.foo = foo;
   }

   public String getFoo() {
       return this.foo;
   }

   public void setFoo(String foo) {
       this.foo = foo;
   }

   public boolean equals(Object o) {
       if (o == this) {
           return true;
       } else if (!(o instanceof SomeClass)) {
           return false;
       } else {
           SomeClass other = (SomeClass)o;
           if (!other.canEqual(this)) {
               return false;
           } else {
               Object this$foo = this.getFoo();
               Object other$foo = other.getFoo();
               if (this$foo == null) {
                   if (other$foo != null) {
                       return false;
                   }
               } else if (!this$foo.equals(other$foo)) {
                   return false;
               }

               return true;
           }
       }
   }

   protected boolean canEqual(Object other) {
       return other instanceof SomeClass;
   }

   public int hashCode() {
       int PRIME = true;
       int result = 1;
       Object $foo = this.getFoo();
       result = result * 59 + ($foo == null ? 43 : $foo.hashCode());
       return result;
   }

   public String toString() {
       return "SomeClass(foo=" + this.getFoo() + ")";
   }
}

Quite the savings in lines of code! So, how does Lombok go from annotated code to fully-generated compiled code? I recently found out the hard way.

Let’s imagine that the Lombok-annotated class from above is used in the following way in fifty different places in a project:

public class AnExampleClass {
   public void launchSomething(String msg) {
       var someClassObj = new SomeClass(msg);
       System.out.println("Hello, " + someClassObj.getFoo());
   }
}

A task arrives to add the following annotation to the code to a specific class’s method:

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface SomeAnnotation {
   String aValue();
}

The annotation is added to the following class:

public class OtherClass {
   @SomeAnnotation
   public void doSomething() {
       System.out.println("Doing other thing");
   }
}

Suddenly, the Java compiler reports a huge amount of errors, fifty of which contain the following message:

java: constructor SomeClass in class SomeClass cannot be applied to given types;
  required: no arguments
  found:    java.lang.String
  reason: actual and formal argument lists differ in length

What?! This code compiled perfectly well before – and your IDE isn’t showing any errors in the code related to the constructor for SomeClass – so why is it failing everywhere now? The reason is due to how Lombok integrates itself into the Java compilation process. While a developer might perceive Java compilation to be one straightforward operation of translating Java source code into bytecode, what actually occurs is an iterative process. In each compilation “round”, the Java compiler analyzes the compilation errors that it has accumulated and determines whether these errors are recoverable. If they are all recoverable, then the compiler proceeds. This is how Lombok is able to make otherwise “incorrect” source code survive the Java compiler’s initial pass: the errors related to the code that Lombok has yet to generate is deemed recoverable, and Lombok then generates the necessary code to satisfy the compiler in the subsequent rounds. And if the compiler has encountered an unrecoverable error? It stops the process and prints out *all* errors – both the errors that were unrecoverable and the errors that could have been recoverable – thus making it looks like Lombok has broken. Admittedly, this took a lot more time to debug than the Python issue: it was necessary to run the Java compiler in debug mode; identify a suitable point for inserting a breakpoint in the compiler code, and stepping through the compilation process until the eventual culprit was discovered in the code in OtherClass.

java: annotation @SomeAnnotation is missing a default value for the element 'aValue'

Oops! Just like before, the fix was easy – add a value for aValue in the annotation declaration in OtherClass – after which all the compilation errors disappeared. To be sure, this error message was being printed out ever since the beginning of the issue, but it’s easy to miss such a message when it’s buried amongst all the accompanying Lombok-related error messages that would get printed out as well in a project that heavily leverages Lombok.

Conclusion

On the surface, the two cases described above are quite different:

• One project was written in Python, and the other in Java.
• One issue’s root cause was discovered by enabling the display of whitespace characters, whereas the other required a rather-involved step-through of the Java compiler.

However, what these two issues shared in common was that an error that initially appeared “impossible” was eventually demonstrated to be a quite simple mistake that required a quick fix in the end. It’s easy to start imagining that we’ve encountered some nightmare bug that is being caused by a fault in the programming language, a stick of RAM that needs to be replaced, nasal demons, etc, but the state of the software development industry is much more secure than what our fears might lead us to believe, especially for mature languages like Java and Python that have decades of history behind them. When encountering these types of perplexing issues, it can be well worth the while to relax, pause to reflect – and maybe go grab a rubber duck as well – and reassure yourself that if the bug is causing you to question whether 2+2=4, the actual problem is likely something far simpler and easier to remedy in the end.