I've been working on a new project, called Entertainer, which is aiming to be similar to MythTV, but with a featureset devoted more to end users than MythTV is. It's got a pretty slick interface (although some of the interface code is based on an alpha library right now), and has been quite educational for me, especially since I've been working specifically with the gstreamer python bindings.

Recently, I've been working on a VideoThumbnailer for Entertainer using gstreamer. While I worked on it, another developer broke the actual GUI code (or rather, a widget from a library Entertainer's GUI depends on broke), which completely halted me from testing it (Entertainer is pre-alpha). This was okay, because I had only been reading documentation on the Gstreamer, and hadn't quite gotten to code yet. However, before I got to coding, I decided that I needed a way to the thumbnailer, and with the frontend broken, I couldn't very well do that. So I decided to write a unit test.

After stubbing out the unit test, I got to work. Originally, I was using nose and then eventually just creating my own test runner, to eliminate dependencies (we already have a ton of them...). I put a lot of work into the test framework, making sure output was clean and everything.

It was only after all of this, and I was working on the thumbnailer and running the unittest to see if it worked that I realized I was doing test-driven development. It was something I had always read about, but had never done. I will swear by it now. It's made my life so simple. Even now that the GUI in Entertainer is working again, I don't much care to use it to test. I've got a unittest that tells me all I need. If I need more, I'll modify the unittest.

What's even better is that the rest of the Entertainer developers are also getting on the bandwagon. We even have a developer who's never written python before that will be writing the tests for a lot of the code we have. How cool?

If you've ever used IPython, you'll know that it's the cat's meow. It's interactive shell adds some nice features that aren't available in the default python shell, and it's helped me in debugging on countless times. The IPython crew is preparing their IPython 0.8.2 Release. Help out the OSS world by testing the new IPython and reporting bugs into their Trac System. You're not doing anything better this weekend anyway.

You've heard of it. All the cool kids are doing it. If you're not doing it, you're living under a rock. C'mon, just try it. The girls will start noticing you if you do it. You'll start meeting all sorts of cool people, and get invited to all the 1337-est parties. No, I'm not talking about cigarettes. I'm talking about fuzzing.

Fuzzing is the process of providing random data (the "fuzz") to the input of a program. Anywhere that input is received into a program, a fuzzer would allow you to test the robustness of your platform. A network fuzzer would be able to construct a random packet of a given type and throw it to an open port. File fuzzers create random files to be opened in a given program. Hopefully, you get the idea. The concept here is not new, but it seems that there has been an increased buzz for fuzzing in the last few years.

This last weekend at Defcon 15, there was a presentation on a network fuzzer called Funk. Fuzzing for me has never been able to draw me to it the way others have clinged to it. Some of that probably has to do with my lack of interest in hardcore security. However, I was drawn to this fuzzer because it was written in Chicken Scheme. Scheme is a great little language from what I've experienced, but I was having a hard time seeing an actual use for the language. So as I sat in the talk, I checked out that code and took a gander. It was a cool idea, and I've already got a little bit of a contribution patch that I'm preparing to submit to the project.

At the BlackHat Briefings in Las Vegas last week, Mozilla released a Javascript fuzzer designed to help browser developers to "fuzz" their browsers to find security vulnerabilities before releasing their products. The media can spin it how they like, but I am glad to see them opening up their tools, and providing documentation on using them. I especially find it useful because Opera, a competing browser, has now used the tool to find at least four bugs in their browser and release a new build. Hooray for communities!

Let's face it, every piece of software has its bugs. However, in the use case that Mozilla is using their fuzzer (and many other companies use their fuzzers), the fuzzing is equal to some great tests. As a web developer by day, I quickly learned that input should ALWAYS be sterilized, no matter how much you "trust" the users of your platform. It's hard, however, for me to find attack vectors in my own software, mostly because I've designed it around one use case, i.e. the one given to me by the "higher-ups." In my mind, I would say "If only I had a tool that would give me some real world use cases," in hopes to cover both stupid users (creating good error reporting) and malicious users (sterilizing the data or failing silently). Creating a good fuzzing tool would fit this bill exactly.

I'm sold on fuzzing now

I've been intrigued with the concept of testing software for quite some time. Unfortunately, the team I work with at my employer is so small that we hardly have time to write tests for everything we've needed. This means that my experience with testing software has been solely on personal, non-critical projects. I've read books and blog posts and entire mailing list threads on the topic of application testing with fascination. The one thing I've seen lacking is information on testing web applications. It seems the mentality of web applications is that since web applications are all written in Java, we can just use any of the Java testing suites. I hope you see the problem....

Yesterday, I stumbled upon Grig Gheorghiu's blog post about his new O'Reilly e-book An Introduction to Testing Web Applications with twill and Selenium which I went out and purchased, mostly because the price was right. I figured if the book sucked, it was a $10 loss. I'd never purchased an e-book before, but it was a subject that appeared to be something I'd definitely be interested in.

I've played with Selenium before, but I didn't find it to be incredibly useful, and I could never get the Selenium Remote Control to work right. I'd heard about twill, but I had never actually experienced it. After having downloaded and written tests in twill, I can see it's exactly what I've been spending a lot of time writing : a DSL to wrap Python's mechanize module for web testing. The limitation I had with twill was with pages that were AJAX driven, because it couldn't interpret Javascript. That's why the combination of twill and Selenium is so great. Because Selenium runs in your browser (it's got a Firefox extension...), it allows the browser to execute the Javascript, thus giving Selenium access to dynamically created DOM elements.

While I haven't quite finished the whole "book" yet, I'd highly recommend it to anyone who's tired of writing new features for their web applications, and fearing that they may be breaking other functionality at the same time. After subscribing to the testing-in-python mailing list, I cannot stress enough the importance of testing, testing, and more testing.