How To Organize Your Tests
It starts out pretty simple. You want to start writing tests, so you create a library. Before you know it, you’ve got a huge suite. Although there’s some order in the chaos, you probably wish there was some way to make it look better, and even more so, make the team work in a consistent manner.
But before we discuss how, let’s ask this: Can better test organization make us more effective?
The harsh answer, forgive me all you organized people, is that it probably won’t. In fact spending time on organization is a waste. Your choice of tools has much more to do with your effectiveness.
Gil, Say It Ain’t So
It is. And why?
Once a test goes green, it becomes transparent to us. That’s because we run it with a huge group of other green tests. If it’s not red, it’s not important.
How does this relate to test organization? Well, since our test framework of choice flags the red tests immediately, and probably lets you navigate to a test in single click, it doesn’t really matter where the test resides. If the tool you’re using is not integrated with your IDE, it will tell you exactly where and in what line the failure happened (and you should get a better integrated tool).
There are a few things that you can do, that can make you more effective, though:
- Separate quick tests from slow tests. Yes, even if they test the same method. You want to get the quickest feedback you can to keep you going. If you have 100 tests which take 2 seconds to run, and 2 tests that take 1 minute to run, and you always run them together, you’re wasting your time. Moreover, you’ve got time to think if these testing stuff really is worth it. Put the slow ones into separate suites so you can run the quick ones more frequently.
- Don’t blindly put all the tests for all the methods in a class into one file. Imagine that few of tests pass and some fail – you want to get the best picture you can about the status in a single glimpse. So put different but related scenarios together. Separate unrelated scenarios to different files.
- Don’t use test inheritance. It may seem a clever object-oriented way of code reuse but when the test is broken, the last thing you want is to start debugging what it actually does, which setup it requires (in another file) and what happens after it. There are other ways to DRY. Or even better, you can sacrifice the DRY principle in terms of readability, and repeat yourself.
If you’re using Isolator v7, you don’t even have to bother with the first two, since it’s doing it for you automatically: It runs the quick tests automatically, and if a bug occurs, it shows all the tests (both passing and failing) related to the offending code. The third one is still up to you, but if you don’t like to shoot yourself in the foot anyway, you don’t need to worry about test organization at all!