There’s been a lot of talking going on on the web lately. About which mocking framework is the best. EasyMock and Mockito are frequently compared. Especially since Mockito is greatly inspired (and is reusing the same mocking code under the hood) by EasyMock with some tweaks in the syntax. Sadly, while reading comparisons, I’ve noticed things that are just plainly untrue. This article is meant to rectify this and give you my opinion on both frameworks. Being EasyMock lead developer, I’m of course biased. Nevertheless, I believe I did, as best as I could, an honest comparison.
EasyMock doesn’t mock classes by itself
This is false. Since EasyMock 3, standard EasyMock can now mock both classes and interfaces. It used to be true for historical reasons as there were EasyMock and EasyMock Class Extension. The class extension now only exists for backward compatibility reasons (which is an empty shell delegating to EasyMock).
EasyMock code is longer because you need to call replay()
Come on… Like if calling one more method would kill your productivity… And I always thought that it was bringing a nice separation between the preparation phase and actual testing phase. I will agree on one thing though. It used to be annoying to have to replay every single mock (
replay(mock1, mock2, mock3)). That’s why
EasyMockSupport was made. Using it will now require you to call
replayAll() and that’s it.
Beside that, the original statement is a bit fallacious. Mockito requires you to add a bunch of
verify clauses that would have already been included in the EasyMock
expect calls. Here’s an example*:
EasyMock (using EasyMockSupport)
List mock = createNiceMock(List.class); // My business code needs these expect(mock.get(0)).andStubReturn("one"); expect(mock.get(1)).andStubReturn("two"); mock.clear(); replayAll(); // Actual business code someCodeThatInteractsWithMock(mock); // Make sure everything was call as expected verifyAll();
List mock = mock(List.class); // My business code needs these when(mock.get(0)).thenReturn("one"); when(mock.get(1)).thenReturn("two"); // Actual business code someCodeThatInteractsWithMock(mock); // Make sure everything was call as expected verify(mock).get(0); verify(mock).get(1); verify(mock).clear();
You cannot « spy » with EasyMock
Mockito has a nice feature called spying. You create a real object (not a mock) and spy it. It basically means that the object will behave as usual but that all the method calls will be recorded allowing to verify them after. Here’s an example:
List list = new LinkedList(); // real object List spy = spy(list); // spy wrapper //optionally, you can stub out some methods: when(spy.size()).thenReturn(100); //this method call is spied spy.add("one"); // since a real add was done, get(0) will return "one" assertEquals("one", spy.get(0)); //size() method was stubbed and to return 100 assertEquals(100, spy.size()); //optionally, you can verify that add was called as expected verify(spy).add("one");
In EasyMock there’s no spying per se. However, I tried to think about the use-cases and it brought me to the conclusion that by using a subset of EasyMock capture, partial mocking and delegation features, you should be able to cover them all. It might be more cumbersome though. But I think the use-cases are quite rare. I’m not against adding a spy feature if proved wrong.
About the example above, we need to think about the purpose of this test. Let’s say that in this case we have some old code for which we want to make sure the
add method is called with the right parameter but we want to keep its usual behavior. This is quite rare but could happen. We would do:
// Real object List real = new LinkedList(); // create the mock List list = createMock(List.class); // spy the parameter and perform the normal behavior Capture c = new Capture(); expect(list.add(capture(c))).andDelegateTo(real); replayAll(); // the actual test list.add("one"); // get will return what was expected assertEquals("one", real.get(0)); // check the capture (will throw an error if nothing was captured) assertEquals("one", c.getValue()); // this is unnecessary since checking the capture is enough verifyAll();
Better void handling in Mockito
The problem with void method is that they do not return anything. So you cannot do
expect(myVoidMethod()) since it won’t compile. Both EasyMock and Mockito won’t require you to « expect » anything since you most of the time don’t need to return anything anyway. You will just do
It’s wrong to think that you need to call
expectLastCall() with EasyMock. However, it can be done just to make it clear in your code that you are recording. There’s also no need to call
once() since EasyMock expects a single call by default.
// this is enough by itself mock.myVoid(); // no need to do any of this expectLastCall().once();
The main difference is in how you « return » something from a void method. Like throwing an exception. Both frameworks have a special syntax for that. Mockito will use
which is a kind of backward from the usual syntax. EasyMock will instead do
mockedList.clear(); expectLastCall().andThrow(new RuntimeException());
One line longer but keeps the forward syntax (the return is after the call).
Obviously, it amounts to the same thing, as we say in French, « blanc bonnet et bonnet blanc ». And both frameworks are forced to do so for technical reasons.
Mockito errors are clearer
This used to be true. A lot was done in EasyMock 3 to improve error messages. I hope I’ve now filled the gap.
EasyMock breaks tests more often
Yes. That’s the way it should be. That’s my point of view at least. It’s really one of the main differences. With EasyMock, you will be required to expect everything. So your tests will break as soon as you change something. With Mockito, you will verify what you want and only these verifications can break.
What’s the difference? EasyMock forces you to expect everything. Your tests will then break and you will be forced to go back to each test and ask yourself « Hum, was it suppose to break? ». Mockito won’t force you to expect anything. So the test will verify only what the original developer thought would be relevant. But, then, working on some other part of the code, you won’t know a verification is in fact missing. You won’t be forced to have a look at old tests. All tests will stay green and nasty bugs will sneak in. To me, that’s the main difference. A trade-off between a quicker test to write and a test testing the unexpected. Let’s look at an example demonstrating this.
//mock creation List mockedList = mock(List.class); // call the mock in some business code mockedList.add("one"); // the return value isn't used nor specified // mockedList.clear(); // adding this call in the business code WON'T break the test //verification that add() was called with the right parameter verify(mockedList).add("one");
//mock creation List mockedList = createMock(List.class); // We are forced to return a meaningful value expect(mockedList.add("one").andReturn(true); replayAll(); // call the mock in some business code mockedList.add("one"); // the return value isn't used by was specified // mockedList.clear(); // adding this call in the business code will break the test //verification that add() was called with the right parameter // and that no other methods were called verifyAll();
EasyMock example is testing what was intended but it is also doing two things behind the scene.
- Making sure the mock is accurately mocking a List by forcing to return a value
- Preventing the test from working if other calls to the mocks are made by the business code. This will force you to wonder if breaking this test was expected by your refactoring.
Still, I’ve give you one trick to prevent your tests to break all over the place. Think about what is really important. Is it really important that this getter is called only once by the tested method? No? Then record it as a stub. Do I care about this passed parameter? No? Just use the
anyObject matcher. That seems silly but that’s how you get resilient tests. It’s a common mistake. Done usually while fighting to make the test work with the inner implementation of the method instead of thinking about what the tested method should really do (do I hear TDD?)
What do I think of those Mockito guys?
Some of you might think I just hate them. They stole my code and my users. Fortunately, while in other spheres people keep suing each other these days, that’s not the way it goes in the open source world. Everybody is using each other’s ideas and tries to improve them for the benefit of all. The ethic however dictates us to notice each other when we do so. Which they did. I was probably one of the first person aware of Mockito existence and they are not hiding that a lot of code and ideas are coming from EasyMock. These guys did some nice things that made me think. And they have a nice logo. Or course, it ain’t a unicorn world. For sure there is competition. But that’s what pushes us to evolve.
I must confess that EasyMock development has been slower than I would have liked these last two years. Still, EasyMock 3 has brought a lot of improvements. EasyMockSupport, new class mocking API, improved capture, delegation, etc. Also, the merge between EasyMock and EasyMock Class Extension should have been done a long time ago. However, lots of improvements are in the pipe. I’m even looking for help! So feel free to contact me (more on that soon on the EasyMock mailing list. In the future, I would like a better integration with testing frameworks (JUnit, TestNG) and with Hamcrest (coming soon in EasyMock 3.1). Also, some power features to allow crossing the last mocking boundaries like final and private methods would be nice too. And still reduce the boilerplate code and so on. I’m also not closed to add a Mockito kinda API. I still don’t think it’s the best approach to get long term quality but heck… we used to think only interfaces should be mocked…
*: Part of the examples are modified examples taken from the Mockito website