European Testing Conference: Unit Testing the Hard Stuff slides
I’ve just presented my talk on Unit Testing the Hard Stuff at the European Testing Conference.
I hope to share more material in due course, but for the moment, here are my slides (pdf).
Test code needn’t be defensive
In a code review I encountered some test code that looked a bit like this:
var result = await _controller.Resources() as ViewResult; result.Should().NotBeNull(); // ReSharper disable once PossibleNullReferenceException result.Model.Should.BlahBlahBlah();
This is a typical defensive coding pattern whose reasoning goes like this:
- The return type of
_controller.Resources()isTask<ActionResult>. - I need to cast the inner
Resultof thisTaskto aViewResult, as I want to inspect itsModelattribute. - But the
Resultcould be a different subclass ofActionResult, so I had better use a safe cast, just in case. - As I’m using a safe cast, I can’t guarantee that I’ll get any instance back, so I had better do a null check.
- Oh look! ReSharper is complaining when I try to access properties of this object. As I’ve already performed a null check, I’ll turn off the warnings with a comment.
Now, defensive coding styles are valuable when we don’t know what data we’ll be handling, but this is most likely to happen at the boundaries of a system, where it interacts with other systems or, even more importantly, humans.
But in the context of a unit test, things are different:
- We are in control of both sides of the contract: the test and class under test have an intimate and interdependent existence. A different type of response would be unexpected and invalid.
- An attempt to directly cast to an invalid type will throw a runtime error, and a runtime error is a meaningful event within a test. If
_controller.Resources()returns any other subclass ofActionResult, then the fact that it cannot be cast toViewResultis the very information I want to receive, as it tells me how my code is defective.
This means I can rewrite the code like this:
var result = (ViewResult) await _controller.Resources(); result.Model.Should.BlahBlahBlah();
By setting aside the defensive idiom, I’ve made the test clearer and more precise, without losing any of its value.
Identical random values in parallel NUnit test assemblies
We have some NUnit cross-system test assemblies that run in parallel. They upload files and watch to see how they are processed. We were inserting a randomly generated value into the filenames in an attempt to avoid identically named files overwriting each other.
Unfortunately, this didn’t work, and we saw frequent test failures. In particular, we found that filenames generated by one test assembly were often identical to those generated by another.
I wanted to understand why this was happening, so I did some investigation.
We were getting our values fromTestContext.CurrentContext.Random, which is an instance of NUnit’s Randomizer class.
When we look at the implementation of Randomizer, we see this:
static Randomizer()
{
InitialSeed = new Random().Next();
Randomizers = new Dictionary<MemberInfo, Randomizer>();
}
The Randomizer is statically seeded with a value generated by the System.Random class. Because this seed is static, the same sequence of random values is shared by all references to Randomizer within each assembly, producing a sequence of values that are highly likely to be different from each other. However, references to Randomizer in a concurrently running assembly use a different instance and have their own sequence of values with its own seed.
Let’s have a quick look at how System.Random is seeded.
The MSDN documentation tells us:
- The Random() constructor uses the system clock to provide a seed value. This is the most common way of instantiating the random number generator.
And goes on to warn:
…because the clock has finite resolution, using the parameterless constructor to create different Random objects in close succession creates random number generators that produce identical sequences of random numbers.
It would seem that our two parallel test assemblies often start executing within such a short interval of each other, and that NUnit’s Randomizer is seeded with the same value in each assembly, which means the sequences of values are identical.
There is some discussion of introducing a mechanism for controlling the seeding of Randomizer in NUnit, but in the mean time, the solution to our problem was to seed our own System.Random instances, rather than relying on NUnit’s.
Fresh ideas from SoCraTes BE
La Roche-en-Ardenne
I’ve just returned from SoCraTes BE 2017, which took place in the damp and picturesque Ardennes (this leafy picture shows the centre of town!) I come home from every SoCraTes buoyed up my the strength of our community, and full of new ideas and associations.
Here are some of the ideas that excited me this time: (more…)
leave a comment