Testing Your Spring App for Thread Safety

in the following post, i want to show you how to test if your spring application is thread-safe. as an example application, i use the spring petclinic project.

to detect concurrency bugs during our tests, we use vmlens . vmlens traces the test execution and analyzes the trace afterward. it detects deadlocks and race conditions during the test run.

testing

to test the spring project, we parallelize the existing unit tests. the following shows a test method running the existing test shouldfindallpettypes of the clinicservicetests class in parallel:

public class clinicservicetests {
    @test
    public void testmultithreaded() throws interruptedexception 
    {
        testutil.runmultithreaded( new runnable() {
            public void run() {
                try{
                    shouldfindallpettypes();
                }
                catch(exception e)
                {
                    e.printstacktrace();
                }
            }
        }
        , 5);
    }

the testutil.runmultithreaded method runs the runnable with n threads in parallel:

public static void runmultithreaded(runnable  runnable, int threadcount) throws interruptedexception
{
    list<thread>  threadlist = new linkedlist<thread>();    
    for(int i = 0 ; i < threadcount; i++)
    {
        threadlist.add(new thread(runnable));
    }
    for( thread t :  threadlist)
    {
        t.start();
    }
    for( thread t :  threadlist)
    {
        t.join();
    }
}

you can find the source of the class testutil at github here . after running the junit test, we see the following report in vmlens:

analyzing

let us look at one of the races found — accessing the field org.hsqldb.hsqlnamemanager.sysnumber.

the access to the field is locked in the methods org.hsqldb.statementmanager.compile , org.hsqldb.session.execute , and org.hsqldb.jdbc.jdbcconnection.preparestatement , but each thread uses a different monitor.

here is as an example — the org.hsqldb.statementmanager.compile:

public synchronized preparedstatement preparestatement(
          string sql) throws sqlexception {
      checkclosed();
      try {
          return new jdbcpreparedstatement(this, sql,
                  jdbcresultset.type_forward_only,
                  jdbcresultset.concur_read_only, rsholdability,
                  resultconstants.return_no_generated_keys, null, null);
      } catch (hsqlexception e) {
          throw jdbcutil.sqlexception(e);
      }
  }

the problem is that the synchronization happens on the preparedstatement and session, which is created for each thread, while hsqlnamemanager is shared among all threads. that can be seen in the method org.hsqldb.table.createprimarykey:

public void createprimarykey(hsqlname indexname, int[] columns,
                                boolean columnsnotnull) {
       if (primarykeycols != null) {
           throw error.runtimeerror(errorcode.u_s0500, "table");
       }
       if (columns == null) {
           columns = valuepool.emptyintarray;
       }
       for (int i = 0; i < columns.length; i++) {
           getcolumn(columns[i]).setprimarykey(true);
       }
       primarykeycols = columns;
       setcolumnstructures();
       primarykeytypes = new type[primarykeycols.length];
       arrayutil.projectrow(coltypes, primarykeycols, primarykeytypes);
       primarykeycolssequence = new int[primarykeycols.length];
       arrayutil.fillsequence(primarykeycolssequence);
       hsqlname name = indexname;
       if (name == null) {
           name = database.namemanager.newautoname("idx", getschemaname(),
                   getname(), schemaobject.index);
       }
       createprimaryindex(primarykeycols, primarykeytypes, name);
       setbestrowidentifiers();
   }

line 20 shows that the namemanager is part of the database object that is shared among all threads. the race happens in the method org.hsqldb.hsqlnamemanager.newautoname, where the field sysnumber is read and written by the two threads:

public hsqlname newautoname(string prefix, string namepart,
                            hsqlname schema, hsqlname parent, int type) {
    stringbuffer sb = new stringbuffer();
    if (prefix != null) {
        if (prefix.length() != 0) {
            sb.append("sys_");
            sb.append(prefix);
            sb.append('_');
            if (namepart != null) {
                sb.append(namepart);
                sb.append('_');
            }
            sb.append(++sysnumber);
        }
    } else {
        sb.append(namepart);
    }
    hsqlname name = new hsqlname(this, sb.tostring(), type, false);
    name.schema = schema;
    name.parent = parent;
    return name;
}

in line 13, the field sysnumber is incremented by ++. the operation ++ is not one atomic operation, but 6 bytecode operations:

aload 0: this
dup
getfield counter.count : int
iconst_1
iadd
putfield counter.count : int

if two threads execute this in parallel, this might lead to a scenario where both threads read the same value and then both increment the value, leading to duplicate values. and since the field sysnumber is used to generate the primary key, the race condition leads to duplicated primary keys.