I've been posting a lot about test driven development recently, and I got an email from a reader who asked me to go into detail on how I've been going about discovering and working with tests. I think this type of post would have helped me develop the "test first" mindset so much earlier in my career, so I'm sitting down to actually detail exactly how I'm learning about this world that I've always heard about, but only now have visited. I guess an alternate title for this post could be "Python Unit Testing 101"

Because I try to spend at least 10 hours a week on Entertainer, and it's already got a pretty good test base that I have lots of experience in (because I wrote a large chunk of it), I think it would in everyone's best interest if I used a new Entertainer test as an example. If you would like, you can check out the code and see exactly what I've been doing by looking in trunk/src/tests. To run the tests, just navigate to the test directory, and type ./run_tests.py You'll see the tests grind away, telling you which ones passed and which ones didn't (hopefully, all of them are passing when you check out the code...)

Abstraction - The Seed of an Idea

I've been maintaining (or trying to maintain) the thumbnailers, ImageThumbnailer and VideoThumbnailer. They both inherit from an empty abstract class Thumbnailer. Recently, when trying to hunt down a bug, I noticed similar code in both VideoThumbnailer and ImageThumbnailer that could easily be rafactored to an already existing Thumbnailer class. For reference, I actually found this while hunting down a bug in ImageThumbnailer, and writing a test to make sure the bug stayed squished. As I planned out exactly what the Thumbnailer abstract class needed to look like, I started thinking about the test I needed to write first.

The abstract Thumbnailer class looked like this: thumbnailer.py

Planning - Writing the Test

The most important thing to remember about TDD is that your tests describe the way your code should behave, not the way it does behave. For instance, if you find a place where the code should be refactored to behave differently, write the test that way, make sure it fails in the way you expect, and then you can refactor the code to pass the test. That's what I did with the ThumbnailerTest. Since there wasn't any code/logic in the Thumbnailer class, all the tests technically should fail. But I had a few things in mind in refactoring:

1. The Thumbnailer should have logic to tell what type of thumbnail it will create, so it can set it's destination path properly. The types are determined by what folders exist in the thumbnails config directory (currently image, video, and recording). It should throw an exception if it's an invalid type.
2. The Thumbnailer should only try to thumbnail an image or video that exists. It should throw an exception otherwise.
3. It should make sure that the classes that should be overridden by the child classes exist. I'll use a fun little technique for this that I learned only recently.

Obviously, I now have my tests. Let's look at item one. I like to break that test up into two tests, one that tests that each type can be properly parsed and recognized as valid, and one test that throws junk data as the type, and ensures that it throws an exception. I created my test by inheriting from unittest.TestCase, creating a setUp() method, containing only a debug flag (which you can use to make sure your test is working...), and then I start creating my tests.

The result of these tests is the following: Thumbnailer_test.py

On A Tangent - The Test Runner

Before we run the new test in the test suite, take the time to look at the run_tests.py script. run_tests.py

You'll notice that the script searches through the current directory for python files (except those in the blacklist), and using the unittest.TestLoader and unittest.TestSuite classes, loads the those files as modules, and then runs the TestCase child objects. The output of this script is the results from the tests. It's nothing terribly complicated, but then again, if your tests or test harness are complicated, you probably ought to change the implementation.

An alternative to the script used in Entertainer is python-nose. By installing it, you can run each test file yourself by doing nosetests <filename>. It will automatically find the unittest.TestCase instances in your file, and run them, printing the results to the terminal.

The Test Is Failure

Back to the tests at hand, you'll notice they all fail. This is good! If they passed, something is definitely wrong with your tests. Look at the errors for the failed tests and make sure they are what you expect. For instance, if you're trying to call a method that doesn't exist, make sure you've got an AttributeError exception, etc. If they are failing the way you expect, then you can consider your tests to be working properly. Now your job is to hack on the class you're testing until it passes the tests.

While you're writing this class, hopefully you're starting to see the value of the test. It was when I was working on the VideoThumbnailer that I started realizing how great TDD is. Instead of firing up the UI every time I made a small change to the VideoThumbnailer, I just ran the test to see if it was generating thumbnails. So instead of making a change, firing up the ui, navigating to the videos, and checking for thumbnails (and at the time, the UI was in a great state of flux at the time, and was only working for me every once in a while. Having the tests meant making a change, running the test runner to see if the test passed. No dependence on the UI working or anything of the sort.

Your Code Works Because the Test Says It Does

After a quick iteration of work (because you've got the test to run instead of the use case), you've now got working code. You know it's working because your test is passing. You know your test is working because you wrote it, and watched it fail exactly the way it should.

The Thumbnailer class now looks like this: thumbnailer.py

Now you can feel safe in committing your refactored code. Because I already had tests for ImageThumbnailer and VideoThumbnailer written well, the tests required no changes in order to test the code I removed to the abstract class. The general rule is that your class' interfaces should be simple, and shouldn't have to know about the "guts" of another class. Because it was constructed well, the tests worked without a problem.

As you write tests, remember that you're writing code still. I've seen lots of people write the tests and see them pass, and then work on the code, and the tests fall out of maintainability. When you make drastic changes to your code, make the tests reflect that, or rather, make the tests reflect the changes first. When you have tests, your codebase is 2-4 times the size of the code that actually gets deployed, so you've got more code to maintain. However, hopefully the maintainence of the code gets easier as the testing gets better (and if it doesn't, something is wrong with either you, or your codebase).