Normal Tests

If you’ve used the Objective-C unit testing framework (OCUnit, also know as SenTestKit) , or for that matter any other xUnit testing framework, you’ll be familiar with the basic way it goes:

• define a class that inherits from SenTestCase
• add some test methods which take no arguments, return no results, and have names prefixed by “test”
• at runtime SenTest makes a test suite for each SenTestCase subclass it finds
• it adds each test method that it finds to the suite
• it then runs each suite in turn, running each test once, and reporting the results

Parameterised Tests

The normal tests work great if you want to run each test once, but what if you have a set of test data and you want to run each test multiple times, applying each item of test data to it in turn?

The naive approach is to define lots of test methods that just call onto another helper method supplying a different argument each time. Something like this:

- (void)testXWithDataA { [self helperX:@"A"]; } 
- (void)testXWithDataB { [self helperX:@"B"]; } 

That gets tired quickly, and it doesn’t allow for a dynamic amount of test data determined at runtime.

What you really want in this case is to add the following abilities to SenTest:

• the ability to define parameterised test methods using a similar naming convention to the normal ones
• the ability to define a class method which returns a dictionary of test data
• have SenTest make a sub-suite for each parameterised method we found
• have the sub-suite contain a test for each data item
• iterate the suites and tests as usual, applying the relevant data item to each test in turn

SenTestKit is very flexible, but it’s also a bit obscure in the way it’s written, so it’s not immediately apparent how to achieve these goals. After a bit of investigation though it turns out to be pretty simple, and I’ve figured it out so that you don’t have to!

You can find the following classes as part of a small open source module I’ve created, called ECUnitTests. This module includes some other utilities too, but the thing I’m concentrating on in this blog post is the ECParameterisedTest class.

ECParameterisedTest: How To Use It

• inherit from ECParameterisedTest instead of SenTestCase
• define test methods which are named parameterisedTestXYZ instead of testXYZ (they still take no parameters)
• either: define a class method called parameterizedTestData which returns a dictionary containing data
• or: create a plist with the name of your test class, which will contain the data

At runtime the data method will be called to obtain the data. The names of each key should be single words which describe the data. The values can be anything you like - whatever the test methods are expecting.

To simplify the amount of modification to SenTest, the test methods still take no parameters. Instead, to obtain the test data, each test method uses the parameterisedTestDataItem property.

ECParameterisedTest: How It Works

To make its test suites, SenTestKit calls a class method called defaultTestSuite on each SenTestCase subclass that it finds.

The default version of this makes a suite based on finding methods called testXYZ, but it’s easy enough to do something else.

Here’s our version:

+ (id) defaultTestSuite
{
    SenTestSuite* result = nil;
    NSDictionary* data = [self parameterisedTestData];
    if (data)
    {
        result = [[SenTestSuite alloc] initWithName:NSStringFromClass(self)];
        unsigned int methodCount;
        Method* methods = class_copyMethodList([self class], &methodCount);
        for (NSUInteger n = 0; n < methodCount; ++n)
        {
            SEL selector = method_getName(methods[n]);
            NSString* name = NSStringFromSelector(selector);
            if ([name rangeOfString:@"parameterisedTest"].location == 0)
            {
                SenTestSuite* subSuite = [[SenTestSuite alloc] initWithName:name];
                for (NSString* testName in data)
                {
                    NSDictionary* testData = [data objectForKey:testName];
                    [subSuite addTest:[self testCaseWithSelector:selector param:testData name:testName]];
                }
                [result addTest:subSuite];
                [subSuite release];
            }
        }
        
    }

    return [result autorelease];
}

Most of this is self explanatory, but some key things to note are:

• parameterisedTestData is a class method which returns a dictionary containing the data
• the method as a whole returns one SenTestSuite object
• that SenTestSuite object contains more SenTestSuite objects, which in turn contain the actual tests

To make things simple, we want to use the existing SenTestKit mechanism to invoke the test methods. Since SenTestKit expects test methods not to have any parameters, we need another way of passing the test data to each method. Each test invocation creates an instance of a our class, and we do this creation at the point we build the test suite, so the simple answer is just to add a property to the test class. We can set this property value when we make the test instance, and the test method can extract the data from the instance when it runs.

Obtaining Test Data

To obtain the test data, we’ve added a method parameterisedTestData that we expect the test class to implement.

This method returns a dictionary rather than an array, so that we can use the keys as test names, and the values as the actual data. Having names for the data is useful because of the way SenTestKit reports the results.

Typically it reports each test as [SuiteName testName], taking these names from the class and method. Since we’re going to use the name of the test method for each of our suites, we really need another name to use for each test. This is where the dictionary key comes in.

Where the test data comes from is of course up to you and the kind of tests you are trying to perform. There is a simple scenario though, which is that we want to load it from a plist that we provide along with the test class.

Since we need a default implementation of the method anyway, we can cater for this simple case automatically. We look for a plist with the same name as the test class. If we find it, we load it, and return the top level object from it (expecting it to be an NSDictionary).

+ (NSDictionary*) parameterisedTestData
{
    NSURL* plist = [[NSBundle bundleForClass:[self class]] URLForResource:NSStringFromClass([self class]) withExtension:@"plist"];
    NSDictionary* result = [NSDictionary dictionaryWithContentsOfURL:plist];
    
    return result;
}

An Example

Here’s a fully worked example.

###ExampleTests.m:

@interface ExampleTests : ECParameterisedTestCase
@end

@interface ExampleTests

- (void)parameterisedTestOne
{
	STFail(@"test one with data %@", self.parameterisedTestDataItem);
}

- (void)parameterisedTestTwo
{
	STFail(@"test two with data %@", self.parameterisedTestDataItem);
}

@end

###ExampleTests.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
	<dict>
		<key>NameA</key>
		<string>ValueA</string>

		<key>NameB</key>
		<string>ValueB</string>
	</dict>
</plist>

When run, this should produce something like the following log output (simplified for clarity here):

Run test suite ExampleTests
  Run test suite parameterisedTestOne
    test NameA failed: test one with data ValueA
    test NameB failed: test one with data ValueB
  Run test suite parameterisedTestTwo
    test NameA failed: test two with data ValueA
    test NameB failed: test two with data ValueB

I’m migrating some of my git repositories over from github.com/samdeane to github.com/elegantchaos.

Unfortunately that means a change of URL, and there’s no automatic redirection in place to ease this transition.

I’ll try to edit any URLs on this website to point at the new locations, but apologies if I get it wrong.

A while ago I blogged about an Objective-C logging system I’ve developed, called ECLogging.

Previously it was available on github as part of my ECFoundation framework, but recently I’ve been going through ECFoundation and splitting it up into modules, with each one stored as a separate git repository.

Some of these split-out modules still depend on each other, but luckily ECLogging isn’t one of them, which means that you can now adopt it a bit more easily, since you don’t have to take any other baggage with it.

You can find it on github. You can also find a wiki on there with some documentation in it. The github repo also contains a couple of example projects to help you get started.

In my recent post on OS X Helper Applications, I provided some sample code for installing and launching a helper application.

The host application and the helper communicated using distributed objects (NSConnection), running over mach ports (NSMachPort).

At the time, I was aware of the advice in Apple Tech Note 2083: Daemons and Agents which recommends against using mach ports, but I wanted to get something up and running, and the NSMachPort method was the one that seemed to be best documented.

However, after a lot of searching around and poring over some very sparse documentation, I’ve now got a version of the code working using NSSocketPort with unix domain sockets (AF_UNIX style sockets), which the tech note recommends as a good way to do IPC.

Tracking down all the details for this proved to be pretty hard work, and I had to glean various bits of information from a lot of disparate sources - tech notes, email threads and sample code.

The tough bits included:

• getting NSSocketPort to use unix domain sockets
• then getting NSConnection to work with them
• getting launchd to launch a task on-demand when a unix domain socket is hit
• getting the actual socket out of launchd in the server task

I’ve now updated my sample code to include all this good stuff - hopefully it will help someone else out in the future.

As a result of this the sample is a tiny bit more complex, and I’ve also taken the opportunity to refactor some stuff into utility classes. Because these are pretty useful I’ve pulled in my ECFoundation framework, and located some of them in there. However, the sample still doesn’t need the vast majority of ECFoundation, so I haven’t linked to the framework, I’ve just pulled in the source files that I need.

Check out the github repo for more details.

Whatever this time of year means to you, for us it means a few hard-earned days of rest, and probably far too much eating and drinking!

As a result, things will be quiet round here for the next week or two. If you happen to attempt to get in touch, please forgive us if we’re a bit slower than usual in responding.

If you’re on holiday too, we hope you have a good one! If not… well we hope you have a jolly nice time whilst it’s so quiet!

See you all in 2012 I hope.