There’s been a lot of talking going on on the web lately. About which mocking framework is the best. EasyMock and Mockito are frequently compared. Especially since Mockito is greatly inspired (and is reusing the same mocking code under the hood) by EasyMock with some tweaks in the syntax. Sadly, while reading comparisons, I’ve noticed things that are just plainly untrue. This article is meant to rectify this and give you my opinion on both frameworks. Being EasyMock lead developer, I’m of course biased. Nevertheless, I believe I did, as best as I could, an honest comparison.

EasyMock doesn’t mock classes by itself

This is false. Since EasyMock 3, standard EasyMock can now mock both classes and interfaces. It used to be true for historical reasons as there were EasyMock and EasyMock Class Extension. The class extension now only exists for backward compatibility reasons (which is an empty shell delegating to EasyMock).

EasyMock code is longer because you need to call replay()

Come on… Like if calling one more method would kill your productivity… And I always thought that it was bringing a nice separation between the preparation phase and actual testing phase. I will agree on one thing though. It used to be annoying to have to replay every single mock (replay(mock1, mock2, mock3)). That’s why EasyMockSupport was made. Using it will now require you to call replayAll() and that’s it.

Beside that, the original statement is a bit fallacious. Mockito requires you to add a bunch of verify clauses that would have already been included in the EasyMock expect calls. Here’s an example*:

EasyMock (using EasyMockSupport)

List mock = createNiceMock(List.class);

// My business code needs these
expect(mock.get(0)).andStubReturn("one");
expect(mock.get(1)).andStubReturn("two");
mock.clear();

replayAll();

// Actual business code
someCodeThatInteractsWithMock(mock);

// Make sure everything was call as expected
verifyAll();

Mockito

List mock = mock(List.class);

// My business code needs these
when(mock.get(0)).thenReturn("one");
when(mock.get(1)).thenReturn("two");

// Actual business code
someCodeThatInteractsWithMock(mock);

// Make sure everything was call as expected
verify(mock).get(0);
verify(mock).get(1);
verify(mock).clear();

You cannot « spy » with EasyMock

Mockito has a nice feature called spying. You create a real object (not a mock) and spy it. It basically means that the object will behave as usual but that all the method calls will be recorded allowing to verify them after. Here’s an example:

List list = new LinkedList(); // real object
   List spy = spy(list); // spy wrapper

   //optionally, you can stub out some methods:
   when(spy.size()).thenReturn(100);

   //this method call is spied
   spy.add("one");

   // since a real add was done, get(0) will return "one"
   assertEquals("one", spy.get(0));

   //size() method was stubbed and to return 100
   assertEquals(100, spy.size());

   //optionally, you can verify that add was called as expected
   verify(spy).add("one");

In EasyMock there’s no spying per se. However, I tried to think about the use-cases and it brought me to the conclusion that by using a subset of EasyMock capture, partial mocking and delegation features, you should be able to cover them all. It might be more cumbersome though. But I think the use-cases are quite rare. I’m not against adding a spy feature if proved wrong.

About the example above, we need to think about the purpose of this test. Let’s say that in this case we have some old code for which we want to make sure the add method is called with the right parameter but we want to keep its usual behavior. This is quite rare but could happen. We would do:

// Real object
    List real = new LinkedList();
    // create the mock
    List list = createMock(List.class);

    // spy the parameter and perform the normal behavior
    Capture c = new Capture();
    expect(list.add(capture(c))).andDelegateTo(real);

    replayAll();

    // the actual test
    list.add("one");

    // get will return what was expected
    assertEquals("one", real.get(0));

    // check the capture (will throw an error if nothing was captured)
    assertEquals("one", c.getValue());

    // this is unnecessary since checking the capture is enough
    verifyAll();

Better void handling in Mockito

The problem with void method is that they do not return anything. So you cannot do expect(myVoidMethod()) since it won’t compile. Both EasyMock and Mockito won’t require you to « expect » anything since you most of the time don’t need to return anything anyway. You will just do

mock.myVoid();

It’s wrong to think that you need to call expectLastCall() with EasyMock. However, it can be done just to make it clear in your code that you are recording. There’s also no need to call once() since EasyMock expects a single call by default.

// this is enough by itself
   mock.myVoid();
  // no need to do any of this
  expectLastCall().once();

The main difference is in how you « return » something from a void method. Like throwing an exception. Both frameworks have a special syntax for that. Mockito will use

doThrow(new RuntimeException()).when(mockedList).clear();

which is a kind of backward from the usual syntax. EasyMock will instead do

mockedList.clear();
expectLastCall().andThrow(new RuntimeException());

One line longer but keeps the forward syntax (the return is after the call).

Obviously, it amounts to the same thing, as we say in French, « blanc bonnet et bonnet blanc ». And both frameworks are forced to do so for technical reasons.

Mockito errors are clearer

This used to be true. A lot was done in EasyMock 3 to improve error messages. I hope I’ve now filled the gap.

EasyMock breaks tests more often

Yes. That’s the way it should be. That’s my point of view at least. It’s really one of the main differences. With EasyMock, you will be required to expect everything. So your tests will break as soon as you change something. With Mockito, you will verify what you want and only these verifications can break.

What’s the difference? EasyMock forces you to expect everything. Your tests will then break and you will be forced to go back to each test and ask yourself « Hum, was it suppose to break? ». Mockito won’t force you to expect anything. So the test will verify only what the original developer thought would be relevant. But, then, working on some other part of the code, you won’t know a verification is in fact missing. You won’t be forced to have a look at old tests. All tests will stay green and nasty bugs will sneak in. To me, that’s the main difference. A trade-off between a quicker test to write and a test testing the unexpected. Let’s look at an example demonstrating this.

Mockito

//mock creation
 List mockedList = mock(List.class);

 // call the mock in some business code
 mockedList.add("one"); // the return value isn't used nor specified
 // mockedList.clear(); // adding this call in the business code WON'T break the test

 //verification that add() was called with the right parameter
 verify(mockedList).add("one");

EasyMock

//mock creation
 List mockedList = createMock(List.class);
 // We are forced to return a meaningful value
 expect(mockedList.add("one").andReturn(true);
 replayAll();

 // call the mock in some business code
 mockedList.add("one"); // the return value isn't used by was specified
 // mockedList.clear(); // adding this call in the business code will break the test

 //verification that add() was called with the right parameter
 // and that no other methods were called
 verifyAll();

EasyMock example is testing what was intended but it is also doing two things behind the scene.

1. Making sure the mock is accurately mocking a List by forcing to return a value
2. Preventing the test from working if other calls to the mocks are made by the business code. This will force you to wonder if breaking this test was expected by your refactoring.

Still, I’ve give you one trick to prevent your tests to break all over the place. Think about what is really important. Is it really important that this getter is called only once by the tested method? No? Then record it as a stub. Do I care about this passed parameter? No? Just use the anyObject matcher. That seems silly but that’s how you get resilient tests. It’s a common mistake. Done usually while fighting to make the test work with the inner implementation of the method instead of thinking about what the tested method should really do (do I hear TDD?)

What do I think of those Mockito guys?

Some of you might think I just hate them. They stole my code and my users. Fortunately, while in other spheres people keep suing each other these days, that’s not the way it goes in the open source world. Everybody is using each other’s ideas and tries to improve them for the benefit of all. The ethic however dictates us to notice each other when we do so. Which they did. I was probably one of the first person aware of Mockito existence and they are not hiding that a lot of code and ideas are coming from EasyMock. These guys did some nice things that made me think. And they have a nice logo. Or course, it ain’t a unicorn world. For sure there is competition. But that’s what pushes us to evolve.

Last Thoughts

I must confess that EasyMock development has been slower than I would have liked these last two years. Still, EasyMock 3 has brought a lot of improvements. EasyMockSupport, new class mocking API, improved capture, delegation, etc. Also, the merge between EasyMock and EasyMock Class Extension should have been done a long time ago. However, lots of improvements are in the pipe. I’m even looking for help! So feel free to contact me (more on that soon on the EasyMock mailing list. In the future, I would like a better integration with testing frameworks (JUnit, TestNG) and with Hamcrest (coming soon in EasyMock 3.1). Also, some power features to allow crossing the last mocking boundaries like final and private methods would be nice too. And still reduce the boilerplate code and so on. I’m also not closed to add a Mockito kinda API. I still don’t think it’s the best approach to get long term quality but heck… we used to think only interfaces should be mocked…

*: Part of the examples are modified examples taken from the Mockito website