Mike Pence: “BDD is the shizzit… all the goodness of TDD with less religiosity and more practicality”

BDD-style JavaScript testing:

“Borrowing from Behaviour Driven Development techniques, especially the RSpec framework I’ve added some new features to script.aculo.us™ testing library.”

(Via mir.aculo.us.)

Check it out!

In the fine tradition of Hollywood (who these days is remaking everything from old movies, comic books, and video games in lieu of coming up with original ideas) I am doing a remake of the very first post on my blog (which itself was a repost from my Artima.com blog from Feb 2004). The original post created quite a stir at the time, and lead to many of the ideas that have evolved my understanding of TDD and now BDD.

---

Once upon a time (early 2004) there was a bit of fuss on the testdrivendevelopment Yahoo group about the idea of limiting yourself to one assertion per test method, which is a guideline that others and I offer for TDD work.

An address parser was the example of a situation where it was argued that multiple assertions per test made sense. Date was formatted with one address per line, each in one of the following formats:

1. ADDR1$ADDR2$CSP$COUNTRY
2. ADDR1$ADDR2$CSP
3. ADDR1$CSP$COUNTRY
4. ADDR1$CSP

The poster went on to say:

My first inclination is/was to write a test like this (in Java but you get the idea):

a = CreateObject("Address")
a.GetAddressParts("ADDR1$ADDR2$CITY IL 60563$COUNTRY")
AssertEquals("ADDR1", a.Addr1)
AssertEquals("ADDR2", a.Addr2)
AssertEquals("CITY IL 60563", a.CityStatePostalCd)
AssertEquals("Country", a.Country)

 

They didn’t see how to achieve this with one assertion per test, as there are obviously four things to test in this case. I decided that rather than simply reply, I would write some tests and code to illustrate my view on the matter, and offer a solid response.

For the original post, I chose Squeak Smalltalk and Java. In this remake I’m going to use just Ruby. In the original I said “For the sake of conciseness, I’ll omit any required accessors.” Well.. this time around I’m not going to use any accessors, since I am more & more of the opinion that they are quite simply an idea whose sole purpose is to undermine the very essence of Object-Oriented software. I’ll also be using rSpec rather than an xUnit framework.

So, where to start? Well, it often makes sense to start with something simple to quickly and easily get some code written and working. Then it can be extended and evolved. Here the simplest case is: ADDR1$CSP. There are two requirements in the parsing of this example: that the ADDR1 was recognized, and that the CSP was recognized. Viewed this way, we need two examples.

If we want to avoid accessors (and for this exercise, I do) we need another way to see inside the object were working with. This is an address.. let’s say (for the sake of argument & this exercise) that one of the purposes of having an address is to generate address labels of some sort. We could do that with an address label maker object that would interrogate the address object for it’s contents and use them. A better way is to have an address label builder that the address object gives information to. It would have methods such as street_number=, csp=, etc. This is something that we can use Mocks Objects for in our examples.

So, let’s start with an example for ADDR1:

context "An address with street & csp" do

  specify "should capture street information" do
    builder = mock("builder", null_object => true)
    builder.should_receive(:addr1=).once.with("ADDR1")
    addr = Address.from_string("ADDR1$CITY IL 60563")
    addr.use(builder)
  end

end

 

To get this to pass we need an Address class, a from_string factory method, and a use method.

class Address
  def self.from_string(address_string)
    Address.new
  end

  def use(a_builder)
  end
end

 

This lets our spec run and fail:

An address with street & csp
- should capture street information (FAILED - 1)

1)
Spec::Api::MockExpectationError in 'An address with street & csp should capture street information'
Mock 'builder' expected 'street=' once, but received it 0 times
./address_spec.rb:7:in `'

Finished in 0.003395 seconds

1 specification, 1 failure

 

Now let’s make that work:

class Address
  def self.from_string(address_string)
    addr1, rest = address_string.split('$')
    Address.new(addr1)
  end

  def initialize(addr1)
    @addr1 = addr1
  end

  def use(a_builder)
    a_builder.addr1 = @addr1
  end
end

 

This makes the example work:

An address with street & csp
- should capture street information

Finished in 0.003003 seconds

1 specification, 0 failures

 

That’s well & good. The next test is for CSP.

specify "should capture csp information" do
  builder = mock("builder", :null_object => true)
  builder.should_receive(:csp=).once.with("CITY IL 60563")
  addr = Address.from_string("ADDR1$CITY IL 60563")
  addr.use(builder)
end

 

Resulting in:

An address with street & csp
- should capture street information
- should capture street information (FAILED - 1)

1)
Spec::Api::MockExpectationError in 'An address with street & csp should capture street information'
Mock 'builder' expected 'csp=' once, but received it 0 times
./address_spec.rb:15:in `'

Finished in 0.005866 seconds

2 specifications, 1 failure

 

Address#from_string will need to be extended (and we need to add csp support):

class Address
  def self.from_string(address_string)
    addr1, csp, rest = address_string.split('$')
    Address.new(addr1, csp)
  end

  def initialize(addr1, csp)
    @addr1 = addr1
    @csp = csp
  end

  def use(a_builder)
    a_builder.addr1 = @addr1
    a_builder.csp = @csp
  end
end

 

So. We have two tests for this one situation. Notice the duplication in the tests… the creation of builder & address. This is the context. After refactoring, we have:

require 'address'

context "An address with street & csp" do

  setup do
    @builder = mock("builder", :null_object => true)
    @addr = Address.from_string("ADDR1$CITY IL 60563")
  end

  specify "should capture street information" do
    @builder.should_receive(:addr1=).once.with("ADDR1")
    @addr.use(@builder)
  end

  specify "should capture csp information" do
    @builder.should_receive(:csp=).once.with("CITY IL 60563")
    @addr.use(@builder)
  end
end

 

So, a context that creates the Address instance from the string as well as the mock, and very simple examples that focus on each aspect of that context.

Note that we don’t have any direct expectations in these examples… What’s up? Well, the mock is autoverifying itself at the end of each example, and that is where our expectations are.. on @address’s interaction with the mock.

The next simplest case is the obvious choice for the next context:

context "An address with street, csp, and country" do

  setup do
    @builder = mock("builder", :null_object => true)
    @addr = Address.from_string("ADDR1$CITY IL 60563$COUNTRY")
  end

end

 

This set of tests will include ones for addr1 and csp as before (since that behaviour is required in this new context) as well as a new test for country:

specify "should capture country information" do
  @builder.should_receive(:country=).once.with("COUNTRY")
  @addr.use(@builder)
end

 

As before, an instance variable and such need to be added to the Address class.

This drives Address to evolve:

class Address
  def self.from_string(address_string)
    addr1, csp, country, rest = address_string.split('$')
    Address.new(addr1, csp, country)
  end

  def initialize(addr1, csp, country)
    @addr1 = addr1
    @csp = csp
    @country = country
  end

  def use(a_builder)
    a_builder.addr1 = @addr1
    a_builder.csp = @csp
    a_builder.country = @country
  end
end

 

Now that we are supporting country, we can go back to the first context and add an example specifying that no country information should be provided to the builder:

specify "should not capture country information" do
  @builder.should_not_receive(:country=)
  @addr.use(@builder)
end

 

A slight tweak to Address#use will make this work:

def use(a_builder)
  a_builder.addr1 = @addr1
  a_builder.csp = @csp
  a_builder.country = @country unless @country.nil?
end

 

From here on, the evolution gets a bit more complex, as we add the ADDR2 option to the mix.

The final code and output is:

address_spec.rb

require 'address'

context "An address with street & csp" do

  setup do
    @builder = mock("builder", :null_object => true)
    @addr = Address.from_string("ADDR1$CITY IL 60563")
  end

  specify "should capture street information" do
    @builder.should_receive(:addr1=).once.with("ADDR1")
    @addr.use(@builder)
  end

  specify "should capture csp information" do
    @builder.should_receive(:csp=).once.with("CITY IL 60563")
    @addr.use(@builder)
  end

  specify "should not capture country information" do
    @builder.should_not_receive(:country=)
    @addr.use(@builder)
  end
end

context "An address with street, csp, and country" do

  setup do
    @builder = mock("builder", :null_object => true)
    @addr = Address.from_string("ADDR1$CITY IL 60563$COUNTRY")
  end

  specify "should capture street information" do
    @builder.should_receive(:addr1=).once.with("ADDR1")
    @addr.use(@builder)
  end

  specify "should capture csp information" do
    @builder.should_receive(:csp=).once.with("CITY IL 60563")
    @addr.use(@builder)
  end

  specify "should capture country information" do
    @builder.should_receive(:country=).once.with("COUNTRY")
    @addr.use(@builder)
  end
end

 

address.rb

class Address
  def self.from_string(address_string)
    addr1, csp, country, rest = address_string.split('$')
    Address.new(addr1, csp, country)
  end

  def initialize(addr1, csp, country)
    @addr1 = addr1
    @csp = csp
    @country = country
  end

  def use(a_builder)
    a_builder.addr1 = @addr1
    a_builder.csp = @csp
    a_builder.country = @country unless @country.nil?
  end
end

 

spec output

An address with street & csp
- should capture street information
- should capture csp information
- should not capture country information

An address with street, csp, and country
- should capture street information
- should capture csp information
- should capture country information

Finished in 0.014 seconds

6 specifications, 0 failures

 

Conclusion

So we took a situation that was thought to require multiple assertions/expectations in a test/example and did it in such as way as to have only one per.

The key is that instead of using a single context with a complex (i.e. multiple expectation) example for each situation, we made each of those situations into a separate context. Now each example focuses on a very small, specific aspect of the behaviour dealt with by its context.

I’m convinced writing examples like this is a useful approach. One advantage is that the resulting examples are simpler and easier to understand. Just as important, and maybe more so, is that by adding the specification of the behavior one tiny piece at a time, you drive toward evolving the code in small, controllable, understandable steps.

It also fits better into the context centered approach that is the recommended way to organize your examples. We set up the object being worked on in setup, and wrote examples of it’s behaviour in that particular context in individual specify clauses.

As I was writing this back in early 2004, something clicked. I saw these test methods (as they are in jUnit/test::unit) as specifications of tiny facets of the required behavior. Thus, it made sense to me to be as gradual as possible about it, driving the evolution of the code in the smallest steps possible. Striving for one assertion per test is a way to do that. This epiphany was one of the main drivers in my deepening understanding of what has been called Test Driven Development (and which we are now considering an aspect of Behaviour Driven Development)

If, however, you view test methods as strictly performing verification, then I can see how it might be seen to make sense to invoke some code and then test all the postconditions. But this view is not TDD (and certainly not BDD), and doesn’t buy you all of the benefits that are possible. I contend that central to TDD is this notion of working in the smallest steps possible, both for the finest-grained long-term verification, and for the most flexible design evolution. Furthermore, this is best done by striving to keep tests/examples as small, focused and simple as possible.

Aiming for one expectation each is one way to get there.

Keith Ray posted a comment/followup to my original BDD article here.

Excerpt:

By the way, I HATE saying TDD is ‘not about the testing’. I have to say it now and then because of people not realizing it’s about designing, rather than testing. The fact that the word ‘test’ is in the name just helps confuse the issue. My preference for naming this design technique, is to call it ‘Behavior Driven Design’. Or ‘Behavior-Spec Driven Design’. With BSDD, I could say BSDD is about the behavior-specs, but mostly about driving design.

(Via Keith Ray.)

The state of Test Driven Development

For the latest version of this information go here and get the article that grew out of this post

Test Driven Development (TDD) has made it to prime time. Big companies are paying big money to have their programmers trained in how to do TDD. It’s a popular topic at conferences… agile and otherwise. My book on TDD won a Jolt award. So everything’s rosy, huh? Everyone who’s doing TDD is fully understanding it and getting the full benefit, right?

Fat Chance!

Maybe 10% of the people I talk to really understand what it’s really about. Maybe only 5%. That sucks. What’s wrong? Well… one thing is that people think it’s about testing. That’s just not the case.

Sure, there are similarities, and you end up with a nice low level regression suite… but those are coincidental or happy side effects. So why have things come to this unhappy state of affairs? Why do so many not get it? Well, first a bit of a history lesson. If you look at an old issue of my old Coad Letter issues, there’s some background on TDD:

“XP originally had the rule to test everything that could possibly break. Now, however, the practice of testing in XP has evolved into Test-Driven Development.”

So way back when, they were talking about writing tests. And that’s probably why it has the testing centric vocabulary, and that is why people think it’s about testing! What else would they think when they have to talk about TestCases & TestSuites, & Tests.. and have to name methods starting with “test”. Granted, this is jUnit specific and nUnit doesn’t have these requirements.

The thing is that when the evolution to TDD happened what we ended up with was a different kind of animal… not just a slight tweak on the original. The original XP folks were writing tests for everything that could break, then they started writing the tests first. Eventually we ended up at TDD. But TDD is not the endpoint everyone seems to think it is… it’s just a stepping-stone.

Also, the idea of “unit” is a major problem. First of all it’s a vague term, and second it implies a structural division of the code (i.e. people think that they have to test methods or classes). We shouldn’t be thinking about units… we should be thinking about facets of behaviour.

Thinking about unit testing leads us to divide tests in a way that reflects the structural arrangement of the code. For example, having a text classes and production classes in a 1-1 relationship

That’s not what we want… we want behavioural divisions.. we want to work at a level of granularity much smaller than that of the typical unit test. As I’ve said before when talking about TDD, we should be working with very small, focused pieces of behaviour… one small aspect of a single method. Things like “after the add() method is called with an object when the list is empty, there should be one thing in the list”. The method is being called in a very specific context, often with very specific argument, and with a very specific outcome.

So, there you have it. A fabulous idea… wrapped in packaging that causes people to think from a testing point of view.

Why is this a problem? Let’s think for a minute about how people often think about testing.

Programmers often think “I’m not going to write all those tests.”, “It’s really simple code, it doesn’t need to be tested”, “testing is a waste of time”, or “I’ve done this (loop/data retrieval/functionalty, etc) millions of times.”.

Project managers often think “we test after the code is done”, “that’s what we have a testing person for”, or “we can’t spend that time now”.

So with people thinking about testing, it’s easy to come up with all sorts of negative reactions and reasons not to do it… especially when time gets short and the pressure’s on.

So if it’s not about testing, what’s it about?

It’s about figuring out what you are trying to do before you run off half-cocked to try to do it. You write a specification that nails down a small aspect of behaviour in a concise, unambiguous, and executable form. It’s that simple. Does that mean you write tests? No. It means you write specifications of what your code will have to do. It means you specify the behaviour of your code ahead of time. But not far ahead of time. In fact, just before you write the code is best because that’s when you have as much information at hand as you will up to that point. Like well done TDD, you work in tiny increments… specifying one small aspect of behaviour at a time, then implementing it.

When you realize that it’s all about specifying behaviour and not writing tests, your point of view shifts. Suddenly the idea of having a Test class for each of your production classes is rediculously limiting. And the thought of testing each of your methods with its own test method (in a 1-1 relationship) will have you rolling on the floor laughing.

So what to do?

First stop thinking in terms of tests. Using something like JUnit makes this hard, so we need to start with a new framework for specifying behaviour. Dan North of ThoughtWorks has started the jBehave project to do just this.

Using a behaviour-centric framework that uses behaviour-centric vocabulary and concepts will let you think in terms of specifying the behaviour you want from you code.

A Behaviour Specification Framework

So what does a behaviour specification look like? Well, a first pass will look and work a lot like jUnit since:

1. it works quite well enough
2. everyone is familiar with it

A major difference is vocabulary. Instead of subclassing TestCase, you subclass Context. Instead of writing methods that start with test you start them should, or preferrably you won;t have to worry about a naming pattern so you can choose the most appropriate name. Instead of doing verification with assertions (e.g. assertEquals(expected, actual)) you specify post conditions with something like shouldBeEqual(actual, expected).

In Smalltalk, and likely Ruby, this can be even more natural if we embed the framework into the class library (a common approach in Smalltalk btw). You could write something like: actual shouldEqual: expected or result shouldBeNull or [2 / 0] shouldThrow: DivideByZeroException.

What Now?

As mentioned above, Dan North has started the jBehave project to create a jUnit replacement for behaviour specification that you can download and experiment with now. I’ll be involved in some way with that project, and I’m planning to explore some ideas in the Smalltalk and Ruby space. We’ll see what happens and where it goes. I’ve also been working on a behaviour oriented version of jUnit.

Summary

Does this mean that everything you know about TDD is now useless? Hardly! All the techniques are just as applicable to BDD. Your approaches to specifying behaviours are going to be much the same. You’ll still want to mock things. And so on. What’s changed is the vocabulary and the point of view.

The point of view is really the important thing. How does changing that change the value of the process? Well, for one thing, if you have a lot of shoulds, the should method names make a very clear specification. Like:

• shouldAllowValidUser
• shouldCheckIsValidBeforeUpdate
• shouldUpdateRecordSet
• shouldWriteToUpdateLog

To sum up:

1. The problem I have with TDD is that its mindset takes us in a different direction… a wrong direction.
2. We need to start thinking in terms of behavior specifications, not verification tests.
3. The value of doing this will be thinking more clearly about each behaviour, relying less on testing by class or by method, and having better executable documentation.
4. Since TDD is what it is, and everyone isn’t about to change their meaning of that name (nor should we expect them to), we need a new name for this new way of working… BDD: Behaviour Driven Development.