Everything that is wrong with mocking, bdd, and rspec

Below is a public excerpt from the recently release RSpec book :

module Codebreaker
  describe Game do
    describe "#start" do
      it "sends a welcome message" do
        output = double('output')
        game = Game.new(output)
        output.should_receive(:puts).with('Welcome to Codebreaker!')
        game.start
      end
      it "prompts for the first guess"
    end
  end
end

This example illustrates what is wrong with standard BDD as practiced today . Consider the following correct implementation of the spec above:

module Codebreaker
  class Game
    def initialize(output)
      @output = output
    end
    def start
      @output.puts("Welcome to Codebreaker!")
    end
  end
end

Now consider the following refactorings of the start method:

@output.write("Welcome to Codebreaker!\n")
@output.print("Welcome to Codebreaker!\n")
> "Welcome to Codebreaker!".split.each{|c| output.write(c)}; output.write "\n"
@output 
> "Welcome to Codebreaker!\n"

All of these produce the exact same result, which is sending the string to the output stream. However, all of these refactorings will fail the test suite.

Now consider the following implementation:

@output.puts "Welcome to Codebreaker!"
@output 
> "You smell bad and are also ugly!"

That test passes the spec above, but it also insults our (paying?) users!

One of the primary reasons for having a test suite is so that I can refactor and have the tests validate my changes . If I have to change my tests and my code just to re-write the way I cause the same end result, then my test suite is useless for refactoring.

So now the obvious question is "Well Nick, how would you test it?"

require 'stringio'
output = StringIO.new
game = Game.new(output)
game.start
output.seek(0)
assert_equal "Welcome to Codebreaker!\n", output.read

This test passes for all the implementations listed initially, and will fail for the implementation in which we insult our users.

Think about the difference here. The original example is saying "Make sure the game calls puts with the following argument" whereas my test is saying "Make sure the game outputs the following string to the output object". The second case is far more useful.

Mocking is a great tool for dealing with stable APIs. However, when introduced in areas of the codebase that have just a small amount of churn, they can cripple the test suite with their brittleness. How can we expect to embrace change when our codebase can't?

RSpec has become so intertwined with mocking and stubbing practices that many people are taking mocking to the extreme and stubbing out their own code.

I will close by deferring to the wisdom of a higher power:

It is the high coupling between modules and tests that creates the need for a mocking framework. This high coupling is also the cause of the dreaded “Fragile Test” problem. How many tests break when you change a module? If the number is high, then the coupling between your modules and tests in high. Therefore, I conclude that those systems that make prolific use of mocking frameworks are likely to suffer from fragile tests.

...

Keeping middle-class test doubles (i.e. Mocks) to a minimum is another way of decoupling. Mocks, by their very nature, are coupled to mechanisms instead of outcomes. Mocks, or the setup code that builds them, have deep knowledge of the inner workings of several different classes. That knowledge is the very definition of high-coupling.

- Uncle Bob Martin on Object Mentor