Mocking: why we picked NSubstitute

I have had discussions with multiple teams I work with that I prefer NSubstitute as the mocking framework we choose and recently this discussion has come up again. So this time I thought I would write down why I prefer it.

The code examples show NSubstitute, Moq and Fake it easy. While my preference is NSubstitute, use this to make up your own mind.

Another point I would like to make is that I think mocking frameworks are generally overused. If you can get away with not using a mock, then do it. But this is separate upcoming post.

Advantages of NSubstitute

Each mocking framework has it's selling points and trade-offs, my reasons for preferring NSubstitute are:

• It works really well with TDD
• You do not have to pay the 'lambda tax' which is present in RhinoMocks, Moq and Fake it easy
• ReSharper code generation works perfectly, this is due to the previous two benefits
• NSubstitute uses c# language features in a really clever way to simplify usage
  • This results in each interaction requiring less code and being easier to read.

I will cover the trade-offs later in this post as well as covering a bit of how NSubstitute works under the covers.

TDD Example

I am a big fan of TDD, I use it where I feel it is appropriate and it serves me well. NSubstitute was designed with TDD in mind and the API shows this. Let's explore a simple example where we want our Quote view model to request a quote.

// empty interface we are TDDing against
interface IQuoteService
{
}

// Create a substitute
var quoteService = Substitute.For<IQuoteService>();
var quoteViewModel = new QuoteViewModel(quoteService);

// Act/When etc
quoteViewModel.RequestQuote();

// Assert/Then
quoteService.Received().SendQuote(Arg.Any<QuoteRequest>());

That is my test, and so far RequestQuote on my view-model and SendQuote on my service do not exist yet.

In the assert section above you can see the usage of NSubstitute for verifying a method has been called. Notice there is no lambda. The Received() method is an extension method on T which returns T, i.e T Received(this T substitute);. Arg.Any<QuoteRequest>() is how we declare argument specifications, we could also do something like this Arg.Is<QuoteRequest>(r => r != null).

Because Received returns the substitute we can use ReSharper (alt + enter) or Visual Studio (ctrl + ,) to generate the method with the signature of void SendQuote(QuoteRequest request).
You can take this a step further and assign this call to something.

Task<QuoteRequestResult> = quoteService.Received()
                                .SendQuote(Arg.Any<QuoteRequest>());

When we use ReSharper of visual studio to generate the method stub they now have the return type of the method and can generate it nicely.

This is very handy when using TDD because Intellisense does not get in my way, if the method I am calling is there it gets picked up and autocompleted. When it doesn't I can generate it when I am finished writing my test.

A lot of this workflow works fine with the other mocking frameworks, I just find it less friction because I can type exactly what I am thinking without remembering the syntax of the particular framework

Comparison of features

Using the above example, this is a comparison of Moq, NSubstitute and Fake it easy. Our quote service now looks like this

interface IQuoteService
{
  Task<QuoteRequestResult> SendQuote(QuoteRequest request);
  int Timeout { get; set; }
  bool IsIntialised { get; }
  event Action Initialised;
}

Setup

// NSubstitute
var quoteService = Substitute.For<IQuoteService>();
quoteService.SendQuote(Arg.Any<QuoteRequest>())
            .Returns(Task.FromResult(new QuoteRequestResult()));

// Moq
var quoteServiceMock = new Mock<IQuoteService>();
quoteServiceMock.Setup(s => s.SendQuote(It.IsAny<QuoteRequest>()))
                .Returns(Task.FromResult(new QuoteRequestResult()));

// Fake it easy
quoteService = A.Fake<IQuoteService>();
A.CallTo(() => quoteService.SendQuote(A<QuoteRequest>._))
 .Returns(Task.FromResult(new QuoteRequestResult()));

What I like/dislike about each:

• + NSubstitute/Fakeiteasy method call is still quoteService.SendQuote, Moq breaks the mock variable up
• + NSubstitute/Moq quoteService is the first thing on the setup line, making scanning code easier
• - Moq/Fakeiteasy use of lambdas creates noise
• + NSubstitute because it is normal method calls/chaining all code formatting works nicely. Lambdas do not format as well when they are complex

From an API point of view, NSubstitute is just really nice for setup.

Assertions/Verification

The other side of it is verifying that your code called the right thing.

// NSubstitute
quoteService.Received().SendQuote(Arg.Any<QuoteRequest>());

// Moq
quoteServiceMock.Verify(q => q.SendQuote(It.IsAny<QuoteRequest>()));

// Fake it easy
A.CallTo(() => quoteService.SendQuote(A<QuoteRequest>._))
 .MustHaveHappened();

• + Fakeiteasy Same syntax in setup/assertions making it easy to learn
• + NSubstitute Very simple and readable
• - Fakeiteasy/Moq Use of lambdas

Not too much to say about this, same points as above really. NSubstitute is just cleaner than the others.

Number of calls

If we want to verify a certain number of calls have been made, here is the syntax

// NSubstitute
quoteService.Received(1).SendQuote(Arg.Any<QuoteRequest>());

// Moq
quoteServiceMock.Verify(q => q.SendQuote(It.IsAny<QuoteRequest>()), Times.Once);

// Fake it easy
A.CallTo(() => quoteService.SendQuote(A<QuoteRequest>._))
 .MustHaveHappened(Repeated.Exactly.Once);

Multiple results

In this we want to throw on the second call

// NSubstitute
quoteService
    .SendQuote(Arg.Any<QuoteRequest>())
    .Returns(
      Task.FromResult(new QuoteRequestResult()),
      Task.Run(() => { throw new Exception(); }));

// Moq
quoteServiceMock
    .Setup(s => s.SendQuote(It.IsAny<QuoteRequest>()))
    .Returns(new Queue<Task<QuoteRequest>(new[]
    {
        Task.FromResult(new QuoteRequestResult()),
          Task.Run(() => { throw new Exception(); }
    }).Dequeue); // Note this is not native support, which is why we need a queue

// Fake it easy
A.CallTo(() => quoteService.SendQuote(A<QuoteRequest>._)
 .ReturnsNextFromSequence(
    Task.FromResult(new QuoteRequestResult()),
    Task.Run(() => { throw new Exception(); })) ;

Events

// NSubstitute
quoteService.Initialised += Raise.Event<Action>();

// Moq
quoteServiceMock.Raise(m => m.Initialised += null);

// Fake it easy
// Note: Currently does not support non-event pattern events. 
// See https://github.com/FakeItEasy/FakeItEasy/issues/30
quoteService.Initialised += FakeItEasy.Raise.With(EventArgs.Empty).Now; 

c# event raising is awkward in some ways, so not much can be done. I still like NSubstitute because the event that is being raised is specified before the mocking framework syntax, but happy with either

Properties

Read/write

// NSubstitute
quoteService.Timeout = 500;
quoteService.Timeout.ShouldBe(500); // Using Shouldly

// Moq
quoteServiceMock.Setup(s => s.Timeout).Returns(500);
quoteServiceMock.Timeout.ShouldBe(500);
// or
quoteServiceMock.SetupProperty(s => s.Timeout);
quoteServiceMock.Timeout = 500;
quoteServiceMock.Timeout.ShouldBe(500);

// Fake it easy
quoteService.Timeout = 500;
// NOTE: Same behavior as NSubstitute
quoteService.Timeout.ShouldBe(500); 

Readonly

// NSubstitute
quoteService.IsInitialised.Returns(true);

// Moq
quoteServiceMock.Setup(s => s.IsInitialised).Returns(true);

// Fake it easy
A.CallTo(() => quoteService.IsIntialised).Returns(true);

Accessing arguments

// NSubstitute
quoteService
    .SendQuote(Arg.Any<QuoteRequest>())
    .Returns(callContext =>
    {
      if (callContext.Arg<QuoteRequest>() == null)
        return Task.Run(() => { throw new NullReferenceException(); });
      return Task.FromResult(new QuoteRequestResult()));
    }

// Moq
var quoteServiceMock = new Mock<IQuoteService>();
quoteService
  .Setup(s => s.SendQuote(It.IsAny<QuoteRequest>()))
  .Returns((QuoteRequest request) =>
  {
    if (request == null)
      return Task.Run(() => { throw new NullReferenceException(); });
    return Task.FromResult(new QuoteRequestResult()));
  });

// Fake it easy
A.CallTo(() => quoteService.SendQuote(A<QuoteRequest>._)
 .ReturnsLazily((QuoteRequest request) =>
 {
    if (request == null)
      return Task.Run(() => { throw new NullReferenceException(); });
    return Task.FromResult(new QuoteRequestResult()));
 })) ;

When accessing arguments Moq I think has a slightly more intuative syntax, but the call context which NSubstitute gives you access to can be quite powerful which solve more complex problems in a nice way.

Mock types

NSubstitute does not support strict mocks which I am very happy about. Strict mocks are a bad idea because they cause brittle tests. It does support partial substitutes.

Recursive mocks and automocking

NSubstitute, fake it easy and Moq all support recursive mocks.

One thing to note is if you are using async/await in your code then NSubstitute and Fake it easy both will automock the task so the await does not throw null reference exceptions which is very handy. So handy I implemented this feature in NSubstitute and helped with Fake it easy.

NSubstitute Tradeoffs

At the beginning of this post, I mentioned that all the frameworks have their selling points and their tradeoffs, personally I think NSubstitute has the best syntax and is the most readable of all the frameworks. This does come at a cost though.

Go back and look at NSubstitutes syntax, then have a think how it must work.

Due to its use of extension methods NSubstitute requires a lot of static state, this means that when you misuse NSubstitute like passing argument specifications into a real object you can cause subsequent calls to NSubstitute to fail. Many of the ways you do this are caught by NSubstitute and it will tell you what has gone wrong, but the main problem is that it blows up after the mistake has happened.

This means a bug in one test can cause the next test to fail. This is bad. Though I have been using NSubstitute for quite a number of years now and this has only happened twice and the problem is normally pretty easy to spot.

To understand how this can happen, let's have a quick look into how NSubstitute works

How Setup Works

var quoteService = Substitute.For<IQuoteService>();
quoteService.SendQuote(Arg.Any<QuoteRequest>())
            .Returns(Task.FromResult(new QuoteRequestResult()));

Let us break down what is going on.

1. Arg.Any<QuoteRequest>() is called, it is a static method.
   • An argument specification is added to a thread static queue
   • Any() returns null
2. SendQuote is called on the substitute, at this point NSubstitute just records this is called
   • The arguments it is called with can be Argument specifications or actual instances. When there is not an actual instance passed into the method call nSubstutes dequeues and argument specification and uses that instead.
3. The Returns extension method is called
   • The last recorded call is set up to return the value passed into returns.
   • The call is removed from the recorded calls list so it does not count when we verify calls later

And that is it, NSubstitute now has a method call which when it matches all argument specifications it returns the provided value.

How using substitutes works

Now we have set up a call our code has to use the substitute.

public async Task RequestQuote()
{
  var request = CreateRequest();
  var result = await quoteService.SendQuote(request);
  ProcessResult(result);
}

Here we call SendQuote, it will match what we setup earlier and return the value we wanted.
The steps are the same as above, except we do not call .Returns() so the call stays recorded.

Note: if you pass argument specifications into a real method call, it will be added to the argument queue but not dequeued until later

How verification works

The last step in our test lifecycle is to verify what has happened.

quoteService.Received().SendQuote(Arg.Any<QuoteRequest>());

1. .Received() extension method is called, it then returns the same substitute in verify mode.
2. Arg.Any<QuoteRequest>()) is called, adding an argument specification

Example of NSubstitute state issue

// Call a normal method, passing a NSubstitute argument specification
quoteViewModel.NotASubstitute(Arg.Any<SomeClass>());

// Call the method on the substitute
quoteService.SendQuote(new QuoteRequest());

// Try to verify
quoteService.Received().SendQuote(Arg.Any<QuoteRequest>());

This will fail, when you call Arg.*, an argument specification is put onto a global queue. So when we called Arg.Any<SomeClass>(), the queue looks like this:

ArgSpecification<SomeClass>

The verification line of code does the following (which explains the failure)
quoteService.Received().SendQuote(Arg.Any<QuoteRequest>()); it does a few things.
1. Tells the quote service mock it is expecting to receive a call which is a verification.
2. Arguments of SendQuote are resolved and added to the queue. Which now looks like [ArgSpec<SomeClass>, ArgSpec<QuoteRequest>]
3. Calls SendQuote, the substitute knows that it is a verification method call. So it will drain the arguments queue and see if they match.
4. The first item is a specification for SomeClass, which will never match an argument of QuoteRequest so your test will fail.

When using NSubstitute normally you will not hit these issues, but the static state is the tradeoff that NSubstitute makes for an improved API.

Summary

You have seen how much of the usage of NSubstitute is simpler and cleaner than the other mocking frameworks, I have also shown you how NSubstitute works to give you an idea of the downsides so you can make an informed decision.