S h o r t S t o r i e s

// Tales from software development

Writing an NUnit test runner

leave a comment »

I would guess that most developers using NUnit use the supplied console or GUI test runners to run their tests and have never considered why they might write their own test runner. That’s certainly true of me but I recently wrote a test application where it wasn’t really possible to use the NUnit test runners but it still made sense to use the classes implemented in the NUnit.Framework assembly.

The requirement was to create a test application for a command line database update tool. The tests would be defined in files that could easily be created and updated using a text editor.

My first attempt used a base class that did all the hard work of performing the test setup, running the database update program, performing the tests, and the test teardown (all defined in the test file) and a set of concrete classes derived from this that did nothing except specify the name of the test.  The tests were executed using the NUnit GUI test runner.

The only problem was that adding a new test meant adding a new concrete class and recompiling the assembly. There were two possible solutions: (1) create a list of test files and then use code generation to create the concrete classes, compile the assembly, and then invoke the NUnit test runner, or; (2) make the base class a concrete class, create an instance of it for each test file, and call its methods using a custom test runner.

I started working on option (2) because it didn’t require much additional work. I quickly realised that no changes were required in the test code that I’d already written. What was really required was simply a test runner that would load the test definition files, one at a time, create an instance of the test class, and then do exactly what the NUnit test runners do: call any methods marked with the TestFixtureSetup attribute, then call methods marked with the Test attribute, and finally the methods marked with the TestFixtureTeardown attribute. It would have to handle exceptions thrown by all these calls but specfically by the Test methods because NUnit assertion failures result in an AssertionException being thrown.

The result is exactly what was needed. Each test file is an XML file with this structure:

<TestDefinition name="Test1" description="Tests update">
 <Setup>
  <SQLCommands>
   <SQLCommand>DELETE FROM person WHERE idno = '9999999999';</SQLCommand>
   <SQLCommand>INSERT INTO person(lastname, forename, idno) VALUES('SMITH', 'JOHN', '9999999999');</SQLCommand>
  </SQLCommands>
 </Setup>
 <Data>
  S 00003 9999999999  ; Select row / Item / value
  U 00002 DAVID       ; Update / Item / value
 </Data>
 <SQLAssertions>
  <SQLAssert name="check_forename" description="Check that forename was updated">
   <Query>SELECT forename FROM person WHERE IDHNO='9999999999';</Query>
   <ResultColumns>
     <ResultColumn name="forename" value="DAVID" />
   </ResultColumns>
  </SQLAssert>
 </SQLAssertions>
 <ErrorFileAssertions>
  <ErrorFileAssertion name="no_errors" description="Check that no errors occurred" doesNotContain="***ERROR***" />
   </ErrorFileAssertions>
 <Teardown>
  <SQLCommands>
   <SQLCommand>DELETE FROM person WHERE idno = '9999999999';</SQLCommand>
  </SQLCommands>
 </Teardown>
</TestDefinition>

 
The <Setup> and <Teardown> elements specify the SQL statements required to create the test scenario and then clean up afterwards. The <Data> element contains the input to the database update program that is executed. The <SQLAssertions> and <ErrorFileAssertions> specify assertions that are applied to the database and the contents of the database update error log, respectively.

These files are easily created and modified by the testers and all that’s required to run the tests is to execute the test runner.

Advertisements

Written by Sea Monkey

August 13, 2010 at 8:00 am

Posted in Development

Tagged with

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: