The other day I was writing some tests for FluentValidation and I noticed that these tests were quite different to those I wrote in 2008 when I first started the project. I thought it would be interesting to explore how my approach to testing changed over the last couple of years.

For this example, we’ll look at the NotNullValidator. In FluentValidation, this is used to specify that a particular property cannot be null:

public class CustomerValidator : AbstractValidator<Customer> {
  public CustomerValidator() {
    RuleFor(x => x.Surname).NotNull();
  }
}

Here are some of the tests for this feature using my old approach:

[Test]
public void NotNullValidator_should_pass_if_property_has_value() {
  var validator = new NotNullValidator();
  var result = validator.Validate(new PropertyValidatorContext(null, new object(), x => "Jeremy"));
  result.Count().ShouldEqual(0);
}
 
[Test]
public void NotNullValidator_should_fail_if_value_is_null() {
  var validator = new NotNullValidator();
  var result = validator.Validate(new PropertyValidatorContext("name", new object(), x => null));
  result.Count().ShouldEqual(1);
}

Here I test this feature by instantiating the NotNullValidator class and passing in a PropertyValidatorContext to simulate input. This is testing the internal API used when calling NotNull.

Here are the same tests using the approach that I use today:

[Test]
public void NotNullValidator_valid_if_property_has_value() {
  var validator = new TestValidator();
  var result = validator.Validate(new Person{Forename = "Jeremy"});
  result.IsValid.ShouldBeTrue();
}
 
[Test]
public void NotNullValidator_not_valid_if_value_is_null() {
  var validator = new TestValidator();
  var result = validator.Validate(new Person{Forename = null});
  result.IsValid.ShouldBeFalse();
}
 
class TestValidator : AbstractValidator<Person> {
  public TestValidator() { 
    RuleFor(x => x.Forename).NotNull();
  }
}

Here I test exactly the same feature but by invoking the public API rather than using the internal API directly.

The end result in the same – I’m testing whether the NotNull validator performs correctly when passed different data. The difference is that these tests are much more resilient and also intention-revealing.

If I were to refactor the internal API (which happens from time to time) then the first set of tests would break and I would have to update them. These tests are brittle. The second set of tests would continue to work and validate the system is working correctly even if I refactor the internal API. 

Additionally, the first set of tests perform their assertions by checking how many items are in the “result” collection. By asserting on the “result.IsValid” property, the second set of tests are far easier to understand. This is helpful to me when I re-visit this code, as I can immediately see what the tests are doing. It also helps other developers looking at the codebase to understand what’s happening without having to know how FluentValidation’s internal API works.