Ruby unit testing is weak as a safety net

Ruby unit testing feels very nice and natural to write with RSpec. Since I started working with Ruby more often that is one of the things I liked the most. However I still love Java (or for this particular example anything with strong typing) and is still my main language, and in many cases is superior to Ruby.

One of those is the value of Unit Tests over the long run and as safety net and aid in refactoring.

Let's see this example in Ruby first:

class Collaborator
  def do_stuff

  end
end

class Unit
  def initialize(collaborator)
    @collaborator = collaborator
  end

  def call_collaborator
    @collaborator.do_stuff
  end
end

describe Unit do
  let(:collaborator) { Collaborator.new }
  subject { Unit.new(collaborator) }

  it 'should call collaborator' do
    expect(collaborator).to receive(:do_stuff)
    subject.call_collaborator
  end
end

That seems like a very simple Unit test. Is making sure that my class under test makes a necesary method call to a collaborator.

I run the test and it passes as expected:

 $ bundle exec rspec
.

Finished in 0.00051 seconds
1 example, 0 failures

Now, sometime later I want to change my collaborator. The collaborator would have its own unit test that I would need to change first of course. However I won't look all over my code base where this method is used (already very difficult to do in Ruby). Then I make my collaborator look like this:

class Collaborator
  def do_the_important_stuff

  end
end

If I rerun my test for Unit this is what I get:

.

Finished in 0.00051 seconds
1 example, 0 failures

Yes, the test still pass even if the method that is supposed to be called in the collaborator doesn't exist anymore!. This would for sure break in production when this branch of code is reached. This can (and need) to be mitigated with more integration tests and not trust so much on the unit tests, but still it leaves the original useless unit test in place.

With Java it is a completely different story and the Unit tests can be trusted a lot more. Here is the same example:

interface Collaborator {
  void doStuff();
}

 public class Unit {
   private Collaborator collaborator;

   public Unit(Collaborator collaborator) {
    this.collaborator = collaborator;
  }

  public void callCollaborator() {
    this.collaborator.doStuff();
  }
}

public class TestEr {

  private Collaborator collaborator = Mockito.mock(Collaborator.class);
  private Unit unit = new Unit(collaborator);

  @Test
  public void shouldCallCollaborator() {
    unit.callCollaborator();
    Mockito.verify(collaborator).doStuff();
  }
}

This test passes as well. But now if go about and change the Collaborator to be

interface Collaborator {
  void doSomeStuff();
}

We will automatically get a compilation error in all places that are using this method. Including our unit test. This will drag us direcly to the error to fix the dependency properly before deploying anything. In this case the unit test has proven to be really valuable as a safety net to capture a problem introduced by a seemingly simple refactor.

Of course the example is super simple, but you can imagine in a big scale application the Ruby problem is not so unrealistic.