TDD can also be a big pain-in-the-ass. Isolating logic for testing can be difficult, especially when you need to rely on code that requires complicated external resources: legacy codebases and hardware being the prime examples. The XNA Framework code, with its substantial functionality abstracting the details of hardware on Windows, the Xbox 360, and Windows Phones, falls squarely into both of those categories.

So, does this mean that it’s impossible to use TDD when working with XNA? Hardly. It can be tough, but there are ways to make it easier. In this post, I’ll talk about how I’ve managed to work with XNA while still giving my code the TDD-loving it deserves / needs.

A lot of the power of the XNA Framework comes from several large, core classes: Game, ContentManager, and GraphicsDevice being the major triumvirate. These classes provide the core update, file management and rendering facilities you need to run a game using XNA. They are also fairly intertwined and take a bit of work to instantiate individually. Even if you can manage to include them in your test framework, their instantiation can be quite resource-intensive. This resource-intensiveness can either make your tests unsuitable for rapid iteration (if you instantiate XNA for each test) or harder to keep independent of each other (if you keep XNA instantiated across all tests, thereby allowing the possibility of tests polluting each other with shared state.)

Of course, the real kicker here is that you don’t even need XNA for a lot of your tests. The XNA Framework itself is relatively mature and tested and you couldn’t fix an XNA bug even if you found one because you won’t have source access. The vast majority of code you would want to develop with TDD will have nothing to do with the large XNA classes. So, my big piece of advice for using TDD with XNA is: don’t apply TDD to XNA!

Not touching XNA using TDD is a pragmatic solution although it may not be the methodologically-correct according to the usual tenets of TDD. Theoretically, you could create a mock object to emulate all the functionality of XNA’s GraphicsDevice. However, that would involve a LOT of effort and probably have a lot of bugs in and of itself. For the vast (VAST!) majority of cases, you can entirely ignore GraphicsDevice’s existence and still get the benefits of TDD, even for code related to graphics. In most cases, what you need to test is not the behavior of GraphicsDevice itself, but the data and commands you send to GraphicsDevice.

What this means is that you can apply TDD to code that needs access to complicated XNA classes, like GraphicsDevice and ContentManager, by designing your code to function (in a minimal fashion) given a null instance of the XNA class they require. You’d be surprised at the amount of code you can test even if the class that does all the “real work” is null.

An example from my own code is my TextureLibrary class. It uses ContentManager to load Texture2Ds given a suitable path, but that is only a small (although crucial) part of what TextureLibrary does. Its main mission is to organize collections of Texture2Ds and Rectangles representing UV coordinates for easy access by the rest of the game. The real trick to implementing this class using TDD was realizing that the part of it’s functionality that actually used ContentManager (the loading of textures) was small and trivially simple to implement. By designing TextureLibrary to function (fully except for texture loading proper) with a null ContentManager passed to it, I could concentrate on implementing and testing all the functionality related to adding and removing texture library entries as well as dealing with edge-cases like repeated adds of the same data, etc. When it came to actually using TextureLibrary in a game, I just had to pass in a valid ContentManager and implement the (trivial) code for loading Texture2Ds. Then TextureLibrary just worked there, pretty much exactly as expected.

So there it is, my big tip for how to use TDD with XNA: don’t use TDD with XNA. Or rather, don’t bother trying to get big, complicated XNA classes to work with TDD. You won’t get that much benefit from doing the “right” thing and using mock objects because most of the code you’ll write won’t actually be all that dependent on those big XNA classes. The way I see it, it’s better to test 80% of your code and release a game then to test 100% of your code but never finish a project.

While it’s easy to track the costs of TDD with a timesheet recording the amount of time spent writing tests versus production code, measuring the benefits in a measurable way takes a bit more creativity. My approach to measuring the benefits of TDD in my own code is to keep a journal of moments of when I think I have reaped benefits due to TDD. This is, of course, a very anecdotal approach to measuring the benefits, but it does provide a basis for tracking effects of TDD that would normally go unmeasured.

To give an idea of how this documentation goes, I’ll present a few entries from my TDD diary, a log of events in which I felt the benefits of having produced my code using TDD:

6/18/2012 Implementing Preallocate on Pool<>
Created trivial test of item counts, expected it to pass
Implemented Preallocate but it didn’t pass
I had made a dumb error by calling the Item allocator directly instead of through the get item function
Caught my dumb mistake, yay!

6/18/2012 Implemented updating list of actors, but found problem when updating actors added other updating actors (like bullets)
Thought: “Great, I probablly need to change the interface for my updating actors list”
Had test for 3 cases related to said list
— Insert updateable increases count
— Inserted updateable updated on list update
— Changing updateability of object in list moves it to update list
With the tests in place, I found that I was able to implement temporary holding lists without changing my interface at all

6/24/2012 I think I can attribute this to TDD: using TDD, my actor update logic ended up being decoupled in its own class instead of being just part of a larger “engine update”
This decoupling was motivated by testing ease, but it made implementing a pause system for my game trivially easy because I could instantiate an actor updater just for
in-game (pause-affected) actors while allowing other actors to be unaffected by pause. Had I not been using TDD, the oversight of not providing a convenient way to tick certain
groups of actors might have meant a lot of re-work.

6/29/2012 I found out that my scenes wouldn’t wait for the previous scene finish transitioning out before activating. I found the bug and fixed it, but had a slight logic error.
My test of standard scene functionality caught it. This way, I fixed it before even having to run the code in game to test.

Given events of TDD benefit like the ones, I can estimate the the amount of that would have been lost had I not been using TDD. This gives me a number that I can compare against the actual time I spend writing tests and help me figure out how well TDD is working out for me.

I can sympathize with a manager who is cautious about adopting TDD, compared to the very palpable up-front costs, the benefits tend to be future-oriented and hard-to-measure. Luckily, I don’t have to worry about convincing managers to let me try TDD for my personal projects. This gives me a chance to give TDD a test-drive (bad pun, I know) and see just how it affects the code I write. Being that I’ve been working a lot with XNA lately, I’ll demonstrate a method to integrate the NUnit unit-testing framework with Visual Studio 2010 Express in order to practice TDD while using XNA.

So, a super brief primer on test-driven development. The main feedback loop in TDD runs roughly as follows:

1. Write a test
2. Run your test suite and (presumably) see your new test fail
3. Write code that will satisfy the test
4. Run your test suite again and (hopefully) see your new test succeed
5. Refactor code as needed
6. Run your test suite to confirm that refactoring hasn’t changed any tested behavior
7. Brag to a co-worker about doing TDD
8. Repeat

Given the above workflow, it becomes clear that I should pretty much run tests whenever I change code and build. In that case, it sounds like I should set up my test runner as a post-build step in my Visual Studio solution.

In order to integrate NUnit with Visual Studio, I first download and install NUnit. Then, I create a solution for my new game that looks something like this:

• MyGame Solution
  • MyGame (an XNA Game project)
  • MyGameLib (an XNA Game Library project)
  • MyGameLibTests (an empty C# project)

I then right-click the projects and tell MyGame and MyGameLibTests to both reference MyGameLib. I will be implementing the majority of my games code in MyGameLib as opposed to MyGame because I want to be able to reference the code in both my test project as well as in the executable itself. Ideally, the MyGame project will contain very little code beyond what is needed to instantiate the code written in MyGameLib and run it as a game.

To actually run the test suite, I open up the property settings for MyGameLibTests and modify the post-build task to run the following:

"C:\Program Files (x86)\NUnit 2.6\bin\nunit-console-x86.exe" /nologo $(TargetFileName)

This will cause Visual Studio to run the command-line version of NUnit and pipe the output into the Build window. If NUnit detects a test failure, it will spit out an error code which will cause Visual Studio to consider the build a failure which will stop me in my tracks until I fix the code so that the test passes (which is what I want.) Because I’m running a 64-bit OS, I need to explicitly tell NUnit to use the 32-bit version of nunit-console since XNA is compiled as 32-bit, hence the x86 in the name of the NUnit executable.

I can run my test suite now but, without tests, that doesn’t mean a whole lot. So, in order to add a little meaning to this whole process, I add a class to MyGameLibTests that looks like ths:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using NUnit.Framework;

namespace MyGameLibTests
{
    [TestFixture]
    public class SomeObjectTests
    {
        [SetUp]
        public void Init()
        {
        }

        [TearDown]
        public void Cleanup()
        {
        }

        [Test]
        public void SomeTest()
        {
        }
    }
}

This is pretty much standard NUnit boilerplate code. The [SetUp] and [TearDown] tags mark functions that will be run before and after each test. The [Test] tag marks a test itself. If I go ahead and toss in a test guaranteed to fail like this one:

[Test]
public void SomeTest()
{
    Assert.IsTrue(false);
}

I can build my solution and confirm that the test suite runs and reports a failure. Great! Also, since I will be adding a lot of test files in the future, I went ahead and made a Visual Studio item template (which you can download here) for creating the NUnit boilerplate. I want there to be as little friction as possible between me and writing new tests.

With my test framework in place, I can start writing real failing tests and then the code that makes them succeed. By tying automated testing to my build process, I guarantee that I will run tests whenever I change code. This should help keep me honest by forcing me to fix errors as soon as they crop up.

Now, if you’ll pardon me, I have some code (and tests) to write.