Pitfall: Spike then Add Tests

A common objection to TDD is “I don’t know where to start?”.

Often we can resolve the uncertainty with a “spike” – we hack some changes, hardcode stuff, break the rules, and build some proof of concept. Sometimes we call this a “tracer buller”.

We’ve manually confirmed that it works!

But we don’t have tests. So we write the tests, commit, and push.

But we wanted to do TDD? What happened?

Where this falls short

On our team, we experimented with writing tests after our spikes, and were consistently disappointed with our results.

  1. The spike code proved hard to test, since we hadn’t built it with unit tests in mind. This was intentional, since we had relaxed our coding practices to make it easy to write the spike. If we coded slowly and deliberately, that would defeat the purpose of the spike.
  2. Did we write enough tests? It was hard to tell. We would add tests until code coverage looked good, but we could still find new scenarios that needed tests that we had overlooked. We prefer to have more confidence that we have properly tested our code, aside from “I think it’s good…?”

Alternative: Spike, Revert, TDD

Our preferred way of coding is strict TDD, where no code statements written unless a failing test asks for the code. We like this because the tests help us exercise our code as we go, and the result is well tested, modular code.

We know that when we practice strict TDD, we have testable code and appropriate test coverage. We’re confident in what we have built, and comfortable maintaining it in the future.

Recently, we had a few team retrospectives where we noticed that we were not satisfied with the outcomes of our coding sessions where we started with spikes. We decided on an experiment: if we don’t practice TDD, we consider it a “spike”. After manually confirming the spike worked, we commit the code with a message that indicates it’s potentially broken. We usually use Arlo’s commit notation, so we’ll use an @ risk attestation (“probably broken”) and an @ change type (“a mix of features, refactoring, etc.”). The commit message might look like this: @ @ SPIKE: figure out what we need to change to display the Whizbangs beneath every Scruudle.

Immediately after committing, we revert the commit. We’re now back to a clean slate, tests are all passing, and we’re ready to practice TDD.

As we work, we refer often to the diff from our @ @ SPIKE commit, which tells us what files need to be changed, and the essential changes to those files. We might copy+paste snippets from the diff to pass a test, especially if there were any algorithm changes, interesting calls to third party libraries, or other short but tricky chunks of code. We are just as likely to read the diff, understand it, and type it out again.

Toss the code, Keep what you learned

When we’re writing scrappy spike code, it’s not good code. We intentionally skip a lot of the rules we usually follow, so we know it’s not very good code. We’re not proud of the code, and we’re not excited to maintain it, so we’re not sad to see it go.

The important output of a spike is what we’ve learned. What parts need to change to add the feature? What incorrect assumptions did we have about the system before we started? How do the third party libraries or other collaborating systems behave? What details about the business domain did we previously overlook that we now need to understand in detail?

We can keep the answers to all of these questions without keeping the code.

An easy way to record what we learned

Instead of taking detailed notes while we’re working on our spike, we use git. Most of what we learned is included in the diff. Sometimes we add extra comments with little insights we obtained – chatty comments that we wouldn’t usually commit, but since we’re about to revert the commit anyway we’re pretty liberal with what we’ll write down. This turns out to be a really lightweight way to capture what we learned. Later that day, when we’re TDDing the code in earnest, we have access to all our notes, organized by file and folder, in our IDE’s git window.

“Once we’ve done it, we’ll know how we should have done it”

One of the odd things about design is that sometimes we get our best ideas only after we see the finished product. I had a university professor who warned us not to harshly criticize each other’s work saying “you should have done Y instead of X”. Our professor pointed out that the only reason it was so clear to us that “you should have done Y” is that our classmate explored all the consequences of X. If we had a blank slate, we probably would also start with X.

One benefit of the spike, revert, TDD approach is that in the process of hacking our first draft, we form opinions about how we ought to do it. After the revert, we have our blank canvas and we can go ahead and build it the way we now know we ought to have built it.

It’s so freeing to hack a first draft, knowing that we won’t be stuck with any of the half-baked structures we put in place to get it working.

Further Reading

Geepaw Hill explores the idea of throwaway spikes in wonderful detail in his blog post An Intro to Spikes