Testing Multithreaded Code

A question that is frequently asked on the jMock¹ users' mailing list² is "how do I use mock objects to test a class that spawns new threads"? The problem is that jMock appears to ignore unexpected calls if they are made on a different thread than that which runs the test itself. The mock object actually does throw an AssertionFailedError when it receives an unexpected call, but the error terminates the concurrent thread instead of the test runner's thread.

Here's a far-fetched example. We have a guard and an alarm. When the guard gets bored, he shouldn't ring the alarm just for kicks.

public interface Alarm {
	void ring();
}

public class testGuardDoesNotRingTheAlarmWhenHeGetsBored() {
	Mock mockAlarm = mock(Alarm.class);
	Guard guard = new Guard( (Alarm)alarm.proxy() );
	
	guard.getBored();
}

Here's an implementation of Guard that should fail the test:

public class Guard {
	private Alarm alarm;
	
	public Guard( Alarm alarm ) {
		this.alarm = alarm;
	}
	
	public void getBored() {
		startRingingTheAlarm();
	}
	
	private void startRingingTheAlarm() {
		Runnable ringAlarmTask = new Runnable() {
			public void run() {
				alarm.ring();
			}
		};
		
		Thread ringAlarmThread = new Thread(ringAlarmTask);
		ringAlarmThread.start();
	}
}

However, the test will pass because the mock Alarm will throw an AssertionFailedError on the ringAlarmThread, not the test runner's thread.

The root of the problem is trying to use mock objects for integration testing. Mock objects are used for unit testing in the traditional sense: to test units in isolation from other parts of the system. Threads, however, by their very nature, require some kind of integration test. Concurrency and synchronisation are system-wide concerns and code that creates threads must make use of operating system facilities to do so.

A solution is to separate the object that needs to run tasks from the details of how tasks are run and define an interface between the two. We can test the object that needs to run tasks by mocking the task runner, and test the implementation of the task runner in integration tests.

In our running example, we can introduce a TaskRunner interface to which the Guard passes tasks that it wants to run instead of explicitly creating new threads.

public interface TaskRunner {
	void start( Runnable task );
}

Our test then looks like:

public class testGuardDoesNotRingTheAlarmWhenHeGetsBored() {
	Mock mockAlarm = mock(Alarm.class);
	TaskRunner taskRunner = ... // What goes here?
	Guard guard = new Guard( (Alarm)alarm.proxy(), taskRunner );
	
	guard.getBored();
}

But how should we implement the TaskRunner that we use in our test? If we pass in a TaskRunner that creates a new thread we'll be back where we started and our tests will still, wrongly, appear to pass. We need to run the task in the same thread as the test runner. The easiest way to do that is to run the task immediately it is started, without spawning a thread at all. To do this we could mock the TaskRunner interface and use a custom stub³ to call back to the task's run method, but that's over-complicating things. It's much easier just to write a class that implements the interface:

public class ImmediateTaskRunner implements TaskRunner {
	public void start( Runnable task ) {
		task.run();
	}
}

Our test then looks like:

public class testGuardDoesNotRingTheAlarmWhenHeGetsBored() {
	Mock mockAlarm = mock(Alarm.class);
	TaskRunner taskRunner = new ImmediateTaskRunner();
	Guard guard = new Guard( (Alarm)alarm.proxy(), taskRunner );
	
	guard.getBored();
}

And the implementation of the Guard and the task runner it uses look like this:

public class Guard {
	private Alarm alarm;
	private TaskRunner taskRunner;
	
	public Guard( Alarm alarm, TaskRunner taskRunner ) {
		this.alarm = alarm;
		this.taskRunner = taskRunner;
	}
	
	public void getBored() {
		startRingingTheAlarm();
	}
	
	private void startRingingTheAlarm() {
		Runnable ringAlarmTask = new Runnable() {
			public void run() {
				alarm.ring();
			}
		};
		
		taskRunner.start(ringAlarmTask);
	}
}

public class ConcurrentTaskRunner implements TaskRunner {
	public void start( Runnable task ) {
		(new Thread(task)).start();
	}
}

Another solution for unit testing would be to run the task in the test runner's thread after the call to guard.getBored() has finished. This might be useful if the Guard class contains try...finally statements that mask test failures caused by the task. Again, we can create a TaskRunner implementation to do this:

public class DelayedTaskRunner implements TaskRunner {
	private List delayedTasks = new ArrayList();

	public void start( Runnable task ) {
		delayedTasks.add(task);
	}
	
	public void runTasks() {
		for (Iterator i = delayedTasks.iterator(); i.hasNext(); ) {
			((Runnable)i.next()).run();
			i.remove();
		}
	}
}

public void testGuardDoesNotRingTheAlarmWhenHeGetsBored
	Mock mockAlarm = mock(Alarm.class);
	DelayedTaskRunner taskRunner = new DelayedTaskRunner();
	Guard guard = new Guard( (Alarm)alarm.proxy(), taskRunner );
	
	guard.getBored();
	taskRunner.runTasks();	
}

Pulling the mechanism for running tasks out of the object that needs tasks to be run can have other benefits beyond easier unit testing. One of the effects that Tim Mackinnon discovered on introducing mock objects into a project was that being forced to test classes in isolation creates "flex points" in the code that, spookily, are exactly where you need them as you evolve the codebase. For example, it would now be trivial to make our Guards use a shared thread pool instead of a ConcurrentTaskRunner.