DanNorth.net

September 10, 2007

Upcoming events

So it’s that time of year again. I’ve got a number of conferences and workshops coming up, ranging over all sorts of topics. I just popped over to Martin Fowler’s site (I’m doing a talk with him this week) and noticed that he has a much more organised setup than me. All his events are in a sidebar and there is a handy link if you want more details. Another idea to go on my to-do pile.

ThoughtWorks Quarterly Technology Briefing

  • Manchester – 12 September 2007 <-- That’s this week!
  • London – 20 September 2007

This is the second instalment in ThoughtWorks series of informal sessions aimed at technologists across the spectrum. Although calling it a technology briefing is a bit inaccurate because the title for this one is “How to Sell Agile to your Organisation”, which has far more to do with the themes of people, risk and change than with anything technological.

This is the talk I’ll be presenting with Martin so I can guarantee a lively session. In his own words: “As I detest selling anything to anyone it will be interesting to see how this talk works out.”

Details and registration info are on the ThoughtWorks website.

RailsConf Europe

  • Berlin – 17-19 September 2007

A lot of Ruby folk seem to have taken to behaviour-driven development. This is almost entirely due to the success of the rspec project, which is in turn due to the enthusiasm and dedication of its developers and the community they have established.

A while back I wrote a story-level BDD framework for Ruby called rbehave which has since been integrated into the rspec project.

I’ll be helping rspec project leads David Chelimsky and Aslak Hellesoy present a workshop entitled A half-day of behaviour-driven development on Rails, where we’ll be demonstrating how rspec helps you write software that is focused on achieving an outcome. It’s at 8:30am on the Monday morning, so make sure you’re there first thing.

Expo-C Roadshow

  • Växjö – 15-16 October 2007
  • Karlskrona – 17-18 October 2007

Expo-C is one of my favourite events. It’s a small conference in south-east Sweden and it seems to attract an audience that really cares about what they are doing. I’ve done two of them now, on very different topics, and on both occasions I was very impressed with the quality of the attendees and the calibre of the other speakers. (I’m usually the only one there who hasn’t written a book.)

This time they are doing two mini-conferences back to back, in Växjö and then Karlskrona, with a tutorial day and a seminar day (six sessions) in each location. I’ll be running full-day tutorials on BDD in Växjö, and Coaching, Communication and Change in Karlskrona. For the seminar I’ll be talking about bridging the communication gap, based on a keynote I gave with Martin Fowler at QCon earlier this year.

I will also be learning how to pronounce “Växjö”.

OOPSLA

  • Montreal – 21-25 October 2007

This will be my first OOPSLA. I’ve heard a lot about it and I’m a bit intimidated. By reputation it seems a bit more “cerebral” than most conferences. It will also be the first time I’ve ever presented JBehave at a conference. No mean feat considering I started writing it at the end of 2003! There’s perpetual beta for you.

My co-presenter is my ThoughtWorks colleague, friend and cybergoth Liz Keogh, the person responsible for getting JBehave to 1.0. I have huge respect for Liz; she manages to combine software with poetry. This isn’t a pretentious metaphor – she actually does combine software with poetry. She ran a haiku workshop at a previous ThoughtWorks away day that many of the attendees nominated as the highlight of their day. She also writes inspiring and inspired blog articles.

I’m only going because I want to see what Liz does when she’s let loose on a roomful of developers. I reckon we’ll end up writing haiku acceptance criteria.

And some others…

There are another couple of events in the pipeline that I will blog about nearer the time (January and February next year). After that I’m going to have a bit of a lie down.

Correction: I got the dates wrong for OOPSLA. Thanks Joshua Graham for putting me straight.
Another correction: My Swedish geography is appalling. Thanks Morgan Persson.

Filed under: BDD, agile, coaching, events, ruby — Dan North @ 10:48 pm

June 17, 2007

Introducing rbehave

rbehave is a framework for defining and executing application requirements. Using the vocabulary of behaviour-driven development, you define a feature in terms of a Story with Scenarios that describe how the feature behaves. Using a minimum of syntax (a few “quotes” mostly), this becomes an executable and self-describing requirements document.

BDD has been around in the Ruby world for a while now, in the form of the excellent rspec framework, which describes the behaviour of objects at the code level. The rspec team has focused on creating a simple, elegant syntax and playing nicely with other frameworks, in particular Rails and the Mocha mocking library.

Inspired by this, I wanted to find a simple and elegant way in Ruby to describe behaviour at the application level. This is a different enough problem that I couldn’t just use rspec. You can skip ahead to the Ruby code if you already know about stories and scenarios. This preamble just sets the scene for the example.

The scenarios that describe a story are made up of “steps” of Givens, Events and Outcomes. These steps can be mixed and matched in different ways to provide different sequences of events. Here is an example:


Story: transfer to cash account
  As a savings account holder
  I want to transfer money from my savings account
  So that I can get cash easily from an ATM

  Scenario: savings account is in credit
    Given my savings account balance is $100
    And my cash account balance is $10
    When I transfer $20
    Then my savings account balance should be $80
    And my cash account balance should be $30

  Scenario: savings account is overdrawn
    Given my savings account balance is -$20
    And my cash account balance is $10
    When I transfer $20
    Then my savings account balance should be -$20
    And my cash account balance should be $10

 

Here we have two givens: one about my savings account and the other about my cash account. We have a single event, namely transferring cash. We have two outcomes, again about the account balances.

This is typical of the scenarios in a story: they revolve around a single event (the feature itself) and prescribe different outcomes for different combinations of givens. Also, notice that the steps themselves are parameterized: the first time my savings account balance is $100, the second time it is -$20, so a story framework needs to accommodate this.

Getting it running

So then I converted the story into a rspec-like structure, preferring simple strings to method names, and do/end blocks rather than classes:


require ‘rubygems’
require ‘rbehave’

Story "transfer to cash account",
%(As a savings account holder
  I want to transfer money from my savings account
  So that I can get cash easily from an ATM) do

  Scenario "savings account is in credit" do
    Given "my savings account balance is", 100
    Given "my cash account balance is", 10
    When "I transfer", 20
    Then "my savings account balance should be", 80
    Then "my cash account balance should be", 30
  end

  Scenario "savings account is overdrawn" do
    Given "my savings account balance is", -20
    Given "my cash account balance is", 10
    When "I transfer", 20
    Then "my savings account balance should be", -20
    Then "my cash account balance should be", 10
  end
end

 

Using rspec to drive the design, I wrote a little framework that would run these scenarios, each one in its own instance of an object (so they were independent), reusing the steps as needed.

So, this only left the problem of the steps themselves. They would have to be defined somewhere else (that I hadn’t figured out yet). Then I thought: each step’s implementation should be pretty trivial, so what would happen if I put the code for each step inline in the scenario? So I ended up with this:


require ‘rubygems’
require ‘rbehave’
require ’spec’ # for "should" method

require ‘account’ # the actual application code

Story "transfer to cash account",
%(As a savings account holder
  I want to transfer money from my savings account
  So that I can get cash easily from an ATM) do

  Scenario "savings account is in credit" do
    Given "my savings account balance is", 100 do |balance|
      @savings_account = Account.new(balance)
    end
    Given "my cash account balance is", 10 do |balance|
      @cash_account = Account.new(balance)
    end
    When "I transfer", 20 do |amount|
      @savings_account.transfer_to(@cash_account, amount)
    end
    Then "my savings account balance should be", 80 do |expected_amount|
      @savings_account.balance.should == expected_amount
    end
    Then "my cash account balance should be", 30 do |expected_amount|
      @cash_account.balance.should == expected_amount
    end
  end

  Scenario "savings account is overdrawn" do
    Given "my savings account balance is", -20
    Given "my cash account balance is", 10
    When "I transfer", 20
    Then "my savings account balance should be", -20
    Then "my cash account balance should be", 10
  end
end

 

For this example it turns out there are no new steps to define in the second scenario, which makes it very easy to read. In general I’m finding that most of the steps get defined in the first one or two scenarios.

Implementing the code

So I saved this to a file as transfer_funds.rb and ran it, and I got two failures:


Running 2 scenarios:
FF

2 scenarios: 0 succeeded, 2 failed

FAILURES:
1) transfer to cash account (savings account is in credit) FAILED
NameError: uninitialized constant Account

2) transfer to cash account (savings account is overdrawn) FAILED
NameError: uninitialized constant Account


 

rbehave prints one character per scenario – a dot means the scenario passed, an F means it failed. At the end of the run it prints a list of the failing scenarios. So this tells me that firstly it runs (hooray!) and secondly both scenarios are failing because I’m missing an Account class. Well I don’t want a whole bunch of failing scenarios that only start to work one at a time as I implement them. That feels too much like broken windows – I’ll get too used to seeing failing scenarios and then I won’t react when workng scenarios start failing. So I introduced the idea of a “pending” scenario, by adding a pending() method:


Given "my savings account balance is", 100 do |balance|
  pending "needs an Account"
  @savings_account = Account.new(bal)
end

 

And I got this:


Running 2 scenarios:
PP

2 scenarios: 0 succeeded, 0 failed, 2 pending

Pending:
1) transfer to cash account (savings account is in credit): needs an Account
2) transfer to cash account (savings account is overdrawn): needs an Account

 

The Ps represent pending scenarios, which means they aren’t working yet but they don’t count as a failure. Then I use rspec to implement an Account with a constructor that takes an initial balance, and give it a transfer_to method that moves money around. Then I remove the pending line:


Running 2 scenarios:
.F

2 scenarios: 1 succeeded, 1 failed, 0 pending

FAILURES:
1) transfer to cash account (savings account is overdrawn) FAILED
Spec::Expectations::ExpectationNotMetError: expected -20, got -40 (using ==)


 

Excellent! I have my first working scenario. Now I just need to add behaviour to the Account class to not transfer money it doesn’t have! But wait a minute, what about documentation? Well I added some listeners to the story runner, so when you run:

transfer_funds.rb --dry-run --format simple

you get:


Story: transfer to cash account
As a savings account holder
  I want to transfer money from my savings account
  So that I can get cash easily from an ATM

Scenario: savings account is in credit
  Given my savings account balance is 100
  Given my cash account balance is 10
  When I transfer 20
  Then my savings account balance should be 80
  Then my cash account balance should be 30

Scenario: savings account is overdrawn
  Given my savings account balance is -20
  Given my cash account balance is 10
  When I transfer 20
  Then my savings account balance should be -20
  Then my cash account balance should be 10

 

This is being generated from the same Ruby code that runs the scenarios themselves.

Next steps

You can gem install rbehave or download the source. There is a sample application (Conway’s Game of Life) in progress in the source code that shows you some of the other features rbehave supports.

rbehave was designed to play nice with other frameworks. In particular, the “world” that each scenario runs in is a module that can be mixed into any object, so you could easily use rbehave with a Rails IntegrationTest or incorporate it into your existing acceptance testing framework. It is also possible, and in fact encouraged, to use rspec in your Outcomes (balance.should == 30).

If you’re interested in using or developing rbehave, please join the mailing lists and let me know how you get on. As a parting thought, rbehave is totally compatible with jruby, so you could start writing your Java acceptance criteria in Ruby and running them in rbehave.

A number of people helped me get rbehave off the ground. In particular I have to thank Niclas Nilsson for kick-starting the whole thing, David Chelimsky (rspec lead) for his sound advice and for adding describe/it to the rspec core, Liz Keogh (jbehave lead) for demanding that steps should take parameters and not taking no for an answer. Also PragDave ran an inspiring meta-programming workshop at QCon that gave me the courage to try this stuff.

Filed under: BDD, ruby — Dan North @ 10:23 pm

February 15, 2007

Monkey business value

So I was hanging out with a bunch of geeks in Switzerland, having one of those late night conversations, and an idea sort of emerged, and the more I thought about it, the more I liked the idea. And then I was thinking that a) I’m useless at following through on ideas and b) I would love someone to take this forwards. So here it is.

Our premise was that the value of automated testing is in its repeatability and low investment (in terms of human effort). However, running the same tests all the time just verifies that the system does a small number of well-known activities.

The value of exploratory testing, on the other hand, is to try all the weird combinations that people might try – aka monkey testing – in order to break the system by using it in unlikely ways. The problem is that exploratory testing requires people, so it’s slow and expensive.

Cue RMonkey (and possibly PyMonkey and JMonkey). RMonkey augments your Ruby acceptance tests, to emulate monkey behaviour. It works like this:

Boring, old-style test:


def test_user_can_login
  web.navigate_to ‘/login’
  web.enter_value :name, ‘bob’
  web.enter_value :password, ’secret’
  web.press_button :login
  assert_that web.current_page, has_title("Welcome")
end

 

Now we monkey it up, with the keywords usually, sometimes and rarely:


require ‘rmonkey’

include RMonkey

def test_user_can_login
  navigate_to ‘/login’

  usually   { web.enter_value :name, ‘bob’ }
  sometimes { web.enter_value :name, ‘kate’ }
  rarely    { web.enter_value :name, random_string() }

  usually { web.enter_value :password, ’secret’  } # sometimes don’t bother

  web.press_button :login
  sometimes { 10.times { web.press_button :login } }  # bored bored bored!

  assert_that web.current_page, has_title("Welcome")
end

 

So now you have automated monkey tests. There are default probabilities for usually/sometimes/rarely (say 80%, 15%, 5%) but that is customizable:

monkey_see :usually => 75, :sometimes => 22, :rarely => 3

The interesting part, of course, is to know what sequence of events leads to a test failure. A more fully-featured RMonkey would keep track of which events it executed and produce an informative narrative if things turned bad.

In the spirit of the infinite monkeys metaphor, this would prove most useful in a soak-testing scenario, whereby thousands of monkeys were hitting an application over an extended period of time. You would want to know that a) mostly it worked, and b) when it didn’t work, which sequence of events caused it to break.

Some obvious applications for RMonkey would be in driving Selenium or Watir, or using JRuby to drive JUnit, MarathonMan, WebDriver or any of the other Java testing frameworks.

The code for rmonkey.rb is simply:


module RMonkey
  LIKELIHOOD = { :usually => 80, :sometimes => 15, :rarely => 5 }
 
  def usually
    yield if rand(100) < LIKELIHOOD[:usually]
  end
 
  def sometimes
    yield if rand(100) < LIKELIHOOD[:sometimes]
  end
 
  def rarely
    yield if rand(100) < LIKELIHOOD[:rarely]
  end
 
  def monkey_see(likelihood)
    LIKELIHOOD.merge(likelihood)
  end
end

 

So anyway, let me know if you would like to get involved in developing RMonkey. I think it’s a really appealing idea and I’d love to see someone do something with it.

Filed under: ruby, testing — Dan North @ 11:34 pm

Powered by WordPress

Creative Commons License
This work is licensed under a Creative Commons Attribution 2.5 License.