The End of Endotesting?

…or why Mockito is my new friend.

So what’s endotesting?

The pioneers of the technique we now know as mocking presented a paper at the XP 2000 conference, where they first introduced the idea to a wider audience. They prefaced it with the quote:

bq. “Once,” said the Mock Turtle at last, with a deep sigh, “I was a real Turtle.”

The Lewis Carroll reference appeals to me because I often cite Humpty Dumpty as the real creator of behaviour-driven development:

bq. “When I use a word, ” said Humpty Dumpty in a rather scornful tone, “it means just what I choose it to mean, neither more nor less.”

These folks – Tim Mackinnon, Steve Freeman and Philip Craig – realised that you could drop in a fake instance of an object that you could get to “fail fast” if it received a method call it wasn’t expecting:

bq. We propose a technique called Mock Objects in which we replace domain code with dummy
implementations that emulate real code. These Mock Objects are passed to the target domain
code which they test from inside, hence the term Endo-Testing.

This approach – endotesting – led to the creation of a number of open source frameworks, most notably JMock (written by Tim and Steve among others) and EasyMock. The basic premise of these frameworks was to make it easy to create a mock object and prime it with a number of expectations. (Your makeCheese method will be called with the parameter @”brie”@. When this happens, return the value BRIE.)

Having set up a mock object with its expectations, you wire it up into a bunch of objects and then poke the outermost object. If all goes well you can interrogate your mock and it will tell you its makeCheese method was called. If it doesn’t go well – well that’s when the magic happens. If you call an unexpected method on a mock it will fail during the test, giving you a stack trace from exactly where the unexpected method call happens. You don’t have to start rummaging through the call sequence trying to work out where it all went wrong, and your test doesn’t carry on running after the unexpected method call to produce a bogus result that doesn’t make any sense.

How we use mocks

Now if you are wiring together a complex group of objects to do something, endotesting-style mock objects are invaluable. I had a situation a while ago where we had a suspect XML message being sent over the wire to an external system. We wrote an integration test where we injected a mock for the external system into a complex object structure – the mock was primed to only accept a valid XML message – and when it failed the stack trace gave us a direct line into the failure. It turned out one of the eight objects wired together was doing an incorrect transformation on part of the XML which meant the outbound message was invalid. By using a mock object in this way we were able to identify exactly where the error was occurring.

However nowadays mocks are mostly used for describing interactions between objects in the context of test-driven development. Your object under test interacts with one or two other objects that you represent by interfaces (roles) that you probably inject into its constructor as mocks. Your test is verifying that the object calls the appropriate methods on its collaborators. In this simplistic scenario, the reason for a failure is usually pretty obvious – you haven’t written the code yet or you are passing the wrong parameters into the method call.

Say you are developing a simple calculation where you expect the answer to be 7 but the code is producing the answer 10, then it isn’t that difficult to work out where the problem is. You don’t need the code to fail as soon as the value of 10 is produced. The fact that the answer is 10 and you expected 7 is enough to triangulate the error. This is classic post hoc state-based testing. Similarly, if you expect your object to call makeCheese on the CheeseFactory and it doesn’t, well you are likely to have a pretty good idea where it doesn’t. So all you really need your mocking library to do is act like a flight recorder black box. You can ask it after the fact: hey CheeseFactory, did my object call your makeCheese method? If it didn’t you probably know why. (And if you don’t it may be an indication that your test is too complicated.)

A more intuitive approach to mocking

This then means that you can treat assertions on method calls just like you treat assertions on state changes, asking the various collaborators what happened after the fact rather than priming your mock objects ahead of time, which in turn means you don’t need to turn your brain inside-out creating a flow like this:

  // Given
  // ... create mocks
  // ... wire up my object

  // Expect (eh?)
  funky.dslWith("stringsForMethods")

  // When
  myObject.makeBrie();

  // Then
  verifyExpectationsSetTwoParagraphsEarlier(); // you remember right?

where it is much more intuitive to write something like:

  // Given
  // ... create mocks
  // ... wire up my object

  // When
  myObject.makeBrie();

  // Then
  verify(CheeseFactory).makeCheese("brie"); // how nice is that?

This, then, is the premise of Mockito. It uses a similar syntax to EasyMock – which means my generics-aware IDE is able to do proper completions and refactorings – but it doesn’t have any of that record/playback nonsense. Nor does it require me to do any of the clever @{{ anonymous constructor }}@ gymnastics that JMock 2 introduced. It just sets up mocks on interfaces (although it allows me to mock concrete classes which I think is a retrograde step – remember kids, mock roles, not objects) and gives me a black box recorder that I can interrogate after the fact.

When I asked Mockito’s author, Szczepan Faber, how he came up with the insight that mocking in the context of TDD was a completely separate problem from endotesting, he replied: what’s endotesting?

It turned out that in order to create a mocking framework to support TDD it helped to have no idea where the encumbent frameworks – JMock and EasyMock – had come from. In the same way that a unit testing framework got co-opted as the enabler for TDD in Java, an endotesting framework became the enabler for method assertions in interaction-based testing. But if you start from the ground up – with a nod to EasyMock and with a pragmatic dose of Java 5 generics – you can get a simple, elegant and intuitive framework for verifying interactions between collaborators. There’s still a place for traditional endotesting in the context of gnarly integration tests, but I’m liking Mockito. I think you will too.

15 comments

  1. Hi Dan,

    Have you seen Brian Takita’s RR framework?

    http://github.com/btakita/rr/tree/master

    It feels more like the second example, although there are some differences as well. I’ve been using it recently and like how it feels.

  2. Really interesting that you should blog this today. Just yesterday I was working on adding some stuff to RSpec to verify expectations after the fact. So you could do

    AccountService.withdraw(50, mock_account)
    mock_account.should have_received(:withdraw).with(50)

    One of my big issues with mocks, in rspec at least, is that they’re so unintuitive in the context of testing. By that I mean, we normally follow an arrange/act/assert pattern, but mocks follow arrange/assert/act/verify-behind-the-scenes. I’ve gotten used to it, but I don’t like it.

    I have a feeling that it’s less of a problem with Ruby because you don’t have nearly as much syntactic noise. So moving the expectations around isn’t quite as profound…but in my opinion, unifying the general format of state-based tests and interaction-based tests is a big win.

    1. Hi Pat.

      That’s great news. I would love to see something like this:

      AccountService.withdraw(50, mock_account)
      ...
      verify(mock_account).withdraw(50)
      

      where the parameter is either a value like 50 or a matcher.

  3. .NET mocking framework moq (http://code.google.com/p/moq/) and others (Rhino Mocks, TypeMock etc) also supports the AAA-style these days, so you can do:

    mock_account.Verify(account => account.withdraw(50));

    and things like

    mock_account.Verify(account => account.withdraw(It.IsAny())); // if you don’t care exactly what the value of the parameter was

  4. When you do TDD you first describe how you want your code to behave with a test, then you make it behave that way.

    When you use mock objects like easymock you first describe how you want your object to behave when called, then you call it.

    It seems that easymock is more similar to test-driven development and mockito more similar to test-afterwards development.

    1. Hi Janis.

      I’ve been using JMock 1 and 2 for a while in the way you describe – writing the example/test ahead of implementing the code. I still do the same thing with Mockito but I find the flow works better. I’ve also found that coaching TDD using Mockito seems to work better than with JMock or EasyMock.

      The only difference between state-based and interaction-based testing is now whether I verify(...) or assertThat(...) things are as I expect them to be. It’s still very much test-driven – the test is written before I implement any code – but the people I’m coaching seem more comfortable with Mockito’s verify afterwards model, and I’m finding it fits the cycle of TDD more naturally.

  5. I wonder. When I see a tool I like to think about its affordances: When we grasp the tool, what action does its form (interacting with our form) suggest to us that we can do with it?

    Putting the expectations after the ‘when’ feels very different from putting them before. Maybe I’m reading too much into this but the expectations-at-the-top model seems to be saying “think about the expected interactions before thinking about the ‘when'” and the expectations-at-the-bottom model seems to say “put in the ‘when’ and then fill in the expectations”

    That turn your brain inside-out feeling isn’t always a bad one and too often “more intuitive” is a technical term meaning “relieves me of the need to think” rather than “makes doing the right think easier.” I wonder exactly which it is in this case.

  6. I finally got to talk to Szczepan about Mockito earlier this year. As you said, he’s very sharp and came with not much baggage. Mockito has raises some issues we should think hard about.

    But, it also became clear that he wrote Mockito to address some weak design and coding habits in his project and that the existing mocking frameworks were doing exactly their job of forcing them into the open. How a team should respond to that feedback is an interesting question.

    In the meantime, I’ve found that I /can/ teach using the existing frameworks if I concentrate on what they were intended for: focusing on the relationships between collaborating objects. I’ve seen quite a few students light up when they get the point. In fact, the syntactic noise in jMock really helps to bring this out, whereas it’s easy for it to get lost with easymock and mockito.

    1. Hi Steve.

      I agree with using JMock as a teaching tool – I’ve had lots of success with it over the years. And I can’t understate the impact it’s had in terms of the wider adoption of mocking, as well as the secondary effects of its matcher library (assertThat and hamcrest).

      I’m interested in how adopting Mockito would “address some weak design and coding habits”. Is it about working around them so you can get work done in spite of the bad habits, or does it provide a more gradual and intuitive path away from them (than expectation-based mocking)?

      An experienced TDDer will usually be equally fluent/expressive/focused using any decent mocking library, but I’ve been finding that novices and advanced beginners “get” mocking more quickly with Mockito than with the expectation-based alternatives. My sample group has been small so it may just be these people in this context. I haven’t been using it for long enough – or across enough different contexts – to have decent empirical evidence yet.

      1. “I’ve been finding that novices and advanced beginners “get” mocking more quickly with Mockito than with the expectation-based alternatives.”

        How are you distinguishing “get mocking” form “gain fluency with the tool”?

  7. […] reply to few interesting blog articles. Dan North became a friend of Mockito and wrote about the end of endotesting. His post sparked few interesting comments. Steve Freeman wrote: But, it also became clear that he […]

  8. Assertion Driven Design (ADD)…

    In all cases of using a unit test framework, no matter the intent of the test methods, one thing is certainly…

  9. […] a couple of years ago with respect to whether using Mockito instead of jMock was bad because it hides design problems that you have with dependencies. Steve Freeman wrote the following comment on Dan's post: But, it also became clear that he wrote […]

  10. Jonathan Woods · ·

    Hi, Dan.

    I feel like some sort of lamer for commenting on a post from pre-history, but it really spoke to an insight I’d had recently re mocks and BDD. That insight too is probably ancient history to you, but since I’ve only just discovered BDD (yes, I know) I’ll take that risk! Love the blog, InfoQ presentations, etc etc, btw.

    I agree that endotesting and top-down development (which is really what BDD and TDD should be called, imho) don’t sit well together. But I think frameworks like JMock still have a place:

    – you mock to implement givens without having to… implement them

    – givens are declarative

    – so if I want to express ‘given there is a flight number BA016′, I’d do that in JMock by saying ‘allowing(flightService).getFlight(‘BA016′); will return new Flight(‘BA016′)'; … in other words, I’m not _requiring_ the call to be made, merely saying what should result if it is

    – presumably, since I’ve been doing the development top-down, FlightService is the only way to get a flight – or at least it’s mutually consistent with all other ways of creating and retrieving them – so this is a valid approach. And I suspect that as you continue the BDD process, you make sure givens are refactored to be shared, which again reinforces the mutual consistency requirement on the implementation

    – even if the codebase hasn’t been developed top-down, and FlightService isn’t the only way to get a flight, but the test passes notwithstanding the fact that the allowed call isn’t made – perhaps because the implementation under test uses a new magically injected TelepathyService – then from the point of view of developing the behaviour I’m interested in here, I don’t really care. I realise I might want to be told that the mechanism I’d expected to be used wasn’t, for other reasons, but for the behaviour here it’s not the prime concern.

    And re patterns like new Expectations() {{}} – I’ve found myself moving towards building expectations instead using an Expectations instance field in my test classes. Especially with a few static imports of Expectations methods, it’s just as expressive and makes for easier refactorings of ‘givens’.

    1. Hi Jonathan.

      I’m not suggesting we shouldn’t have stubs, which is what you’re describing, and Mockito has good support for that. My point is that we can do the actual expectations after the fact, in a black box flight recorder kind of way, which I think leads to more readable tests. And the {{ syntax gymnastics }} with JMock 2 just hurts my brain!

Follow

Get every new post delivered to your Inbox.

Join 486 other followers

%d bloggers like this: