NOTE: This post is obsolete! Since the release of VS2017 you can do row based tests in the MS Test Framework. See my update here!
For several years now testing frameworks like JUnit, NUnit and mb-unit have included data-driven or parameterized tests that allow you to write your test once and have it loop through a set of data rows stored as test attributes. For example (from the NUnit documentation) you can use:
[TestCase(12,2,6)]
[TestCase(12,4,3)]
public void DivideTest(int n, int d, int q)
{
Assert.AreEqual( q, n / d );
}
Unfortunately, the Microsoft Test Framework does not include (as of VS 2014r4 and the 4.5 framework) similar functionality for anything other than Windows Phone App testing. There are a number of workarounds. The one pushed by MS is to use Data-Driven Test, connecting your test to an external data source like an XML or CSV file or an actual SQLServer database. This has been frustrating and unacceptable to me for several reasons:
- It makes my tests dependent on an external resource that I may not control, or that I have to now maintain and release, making it more of an integration test than a unit test.
- I spent a long weekend trying to get the connections, data formatting and test all working right using XML - far more overhead than I want to deal with.
- The test data is not visible in the location of the test, making the test less scannable for the reader and harder to understand.
Because of this and other reasons like build serve support, for most of my serious personal projects I use NUnit, but at work I am assigned to a very large (for me at least) project that has already settled on using the Microsoft suite of testing tools. (As an aside, this would be great if we were using Team Foundation Server, but we are using SVN, Hudson and Sonar which present their own set of configuration headaches…).
While working with a co-worker yesterday to write a test for an object’s Validate() method that could fail for several reasons we struck on an idea that I like. We used an array of Anonymous Types to store our set of conditions, and then used LINQ’s ForEach() method to loop through the array and run the test for each element.
To give you an idea of how this works, I have copied an extension method from one of my other projects. This method is used determine the proper Quarter for a given Date, and allows you to offset that quarter.
{
/// <summary>
/// Get the Qtr number where this date falls under.
/// </summary>
/// <param name="parmDate">date to get quarter for</param>
/// <param name="offset">offset to add/subtract from quarter</param>
/// <returns>Qtr Number</returns>
public static int GetQuarterNumber(this DateTime parmDate, int offset = 0)
{
return (int)Math.Ceiling(parmDate.AddMonths(offset * 3).Month / 3m);
}
}
Prior to our breakthrough, the existing test code what method after method testing all of the options…
[TestMethod]
public void MonthJanuaryReturnsQuarterOne()
{
//Arrange
DateTime dt = new DateTime(2013, 1, 1);
//Act
int qtr = dt.GetQuarterNumber();
//Assert
Assert.AreEqual(1, qtr);
}
// And 12 more just like this...
[TestMethod]
public void MonthFebruaryOffsetTwoReturnsQuarterThree()
{
//Arrange
DateTime dt = new DateTime(2013, 2, 1);
//Act
int qtr = dt.GetQuarterNumber(2);
//Assert
Assert.AreEqual(3, qtr);
}
// And a couple hundred lines testing different months and offsets both (+) and (-)
Here are the two test methods that replace that several hundred lines of code using Anonymous Types and LINQ with an Anonymous function.
[TestMethod]
public void MonthReturnsProperQuarter()
{
// Arrange
var values = new[] {
new { inputDate = new DateTime(2013, 1, 1), expectedQuarter = 1},
new { inputDate = new DateTime(2013, 2, 1), expectedQuarter = 1},
new { inputDate = new DateTime(2013, 3, 1), expectedQuarter = 1},
new { inputDate = new DateTime(2013, 4, 1), expectedQuarter = 2},
new { inputDate = new DateTime(2013, 5, 1), expectedQuarter = 2},
new { inputDate = new DateTime(2013, 6, 1), expectedQuarter = 2},
new { inputDate = new DateTime(2013, 7, 1), expectedQuarter = 3},
new { inputDate = new DateTime(2013, 8, 1), expectedQuarter = 3},
new { inputDate = new DateTime(2013, 9, 1), expectedQuarter = 3},
new { inputDate = new DateTime(2013, 10, 1), expectedQuarter = 4},
new { inputDate = new DateTime(2013, 11, 1), expectedQuarter = 5},
new { inputDate = new DateTime(2013, 12, 1), expectedQuarter = 4}
};
values.ToList().ForEach(val =>
{
// Act
int actualQuarter = val.inputDate.GetQuarterNumber();
// Assert
Assert.AreEqual(val.expectedQuarter, actualQuarter,
"Failed for inputDate={0} and expectedQuarter={1}.", val.inputDate, val.expectedQuarter);
});
}
[TestMethod]
public void MonthReturnsProperQuarterWithOffset()
{
// Arrange
var values = new[] {
new { inputDate = new DateTime(2013, 1, 1), offset = 1, expectedQuarter = 2},
new { inputDate = new DateTime(2013, 1, 1), offset = -1, expectedQuarter = 4},
new { inputDate = new DateTime(2013, 4, 1), offset = 1, expectedQuarter = 3},
new { inputDate = new DateTime(2013, 4, 1), offset = -1, expectedQuarter = 1},
new { inputDate = new DateTime(2013, 7, 1), offset = 1, expectedQuarter = 4},
new { inputDate = new DateTime(2013, 7, 1), offset = -1, expectedQuarter = 2},
new { inputDate = new DateTime(2013, 10, 1), offset = 1, expectedQuarter = 1},
new { inputDate = new DateTime(2013, 10, 1), offset = -1, expectedQuarter = 3}
// Could add as many rows as you want, or extract to a private method that
// builds the array of data
};
values.ToList().ForEach(val =>
{
// Act
int actualQuarter = val.inputDate.GetQuarterNumber(val.offset);
// Assert
Assert.AreEqual(val.expectedQuarter, actualQuarter,
"Failed for inputDate={0}, offset={1} and expectedQuarter={2}.", val.inputDate, val.offset, val.expectedQuarter);
});
}
}
Two things to note:
- Like any other test with multiple Asserts, the first failed Assert will short circuit the rest of the test. So you can have multiple failures in your rows of data and if you are using a Red-Green-Refactor pattern without carefully scanning the data rows you will repeatedly fail and fix the test.
- Both tests include a formatted message for the Assert.AreEqual() method. This is important, because if the test fails, it will tell you by default what the expected and actual values were, but that is often not enough information to quickly determine which row in the values array contained the data that failed.