BDD is like TDD if…

I’ve noticed a number of people recently declaring that BDD really is just TDD, such as Robert Martin on Twitter and Ron Jeffries on the XP list. I can understand where this mindset comes from and I’d like to offer my perspective.

Is BDD the same as TDD? Yes. If you’re a programmer, and your entire team is programmers, and all your stakeholders are programmers and you have a single Subject Matter Expert embedded in the team. Which was true of the Chrysler C3 team and other early XP teams. (Take a look at the original C3 team: 10 world-class programmers and an SME.)

BDD is the same thing but for a broader audience: testers, analysts, project and program managers, multiple SMEs covering multiple interrelated domains. It’s interesting that the BDD == TDD crowd are entirely programmers and necessarily see the world through programmers’ eyes: Bob Martin, Ron Jeffries and others saying “BDD is just TDD” is true in the first approximation of “all delivery people are programmers.” As soon as that precondition fails, BDD becomes a different, bigger thing: it becomes the communication across all of those stakeholders to create a single coherent vision and deliver to that. This is the value of living documentation – and shows why you don’t need it until you do: if you’re in that small team of only-programmers you’ll be fine with only-TDD.

It can be argued that the team shouldn’t be anything other than programmers – in the polyglot roles of tester, analyst, etc. – in which case BDD and TDD again collapse into the same space. But in the kind of organisations and teams that gave rise to and shaped BDD we have to solve the more complex communication problems that are (by definition) beyond the scope of TDD.

I’ve been entirely embedded in small teams of programmers for the last couple of years which is why I’ve taken much more of a back seat in the BDD community. The likes of Liz Keogh, Gojko Adzic and Chris Matts have been where the action is with things like real options and specification-by-example, and folks like Aslak Hellesøy and Matt Wynne figuring out how to manage large numbers of long-lived tests at scale. Liz recently wrote about how shifting the language helps to clarify intent.

Kent Beck describes XP as trying to get business people and technical people on the same page, and I have the same goal for BDD. Don’t forget TDD is one small part of XP. BDD has grown to be broader than s/test/should/, because it’s trying to solve a broader problem.

29 comments

  1. Nice post!
    Can’t you say that BDD == TDD but TDD != BDD? That is, BDD is a super set of TDD. For me BDD is TDD with a shift of focus from the code to the behavior. Then if you write the test in a BDD tool or traditional unit test tool as xUnit doesn’t really matter for me as long as the thing you are focus on is the problem you are trying to solve and not the actual code. Don’t get me wrong here, of course you should care for the code but the test should focus on what you want to do. Also, too many people think that BDD is about using the latest BDD tool out there, and that is where we get wrong a lot of the time. You can use a gherkin tool to write unit test focusing on the unit and not the functionality, but then I don’t think you are doing TDD even though you use your fancy gherkin tool. The same thing goes the other way round, you can still have a mindset of doing BDD even though you are not using the latest BDD tool.

  2. [...] BDD is like TDD if… [...]

  3. Thank you for articulating a very important facet on BDD. Having been through the impacts of BDD on an organization and its culture, I can confirm that BDD the way you are describing is a very different beast than TDD.

  4. Dan,

    First of all I’d like to say that I’m a big fan of the work you’ve done. I think vocabularies and phraseologies that are identified with BDD are quite valuable. I think the BDD tools such as JBehave, Cucumber, and FitNesse have done a lot to help with communications between programmers, stakeholders, SMEs, managers, etc.

    So I don’t want to take anything away from the work you’ve done and the cause you’ve championed.

    From the very beginnings of XP, TDD was about communication. It was, and is, a way for programmers to communicate with each other, and with all the other folks in the team, and connected to the team. Even in the earliest days of TDD we used unit tests for programmer-programmer communication and “functional” tests for programmer-business communication. TDD was never simply for programmers.

    In the years since, we’ve seen dramatic changes in both the form and structure of both unit tests and functional tests. In the latter case, the names have changed dramatically from acceptance tests to integration tests to story tests to simply specifications. We’ve seen tools invade the space like FitNesse, Cucumber, RobotFX, JBehave, etc. In the unit test space we’ve seen the advent of tools like rspec, speclj, and midje which explore different modes of unit-test expression.

    While these tools and changes are significant contributions, they do not represent a change in the basic intent of TDD. TDD was always meant to facilitate communications between everyone on the team.

    So, while BDD is a significant contribution and an important conceptual framework, it does not transcend the original goals of TDD. It fits nicely within those goals. Today it is probably the primary strategy for TDD. But it still is TDD.

    This may seem like an argument over semantics; and it is. But those semantics are important. Lately we’ve seen huge abuses in the name of BDD. For example the Rails community has adopted the strategy of writing hoards of cucumber tests that drive the application through the web server. This leads to slow and fragile tests and breaks many of the rules of TDD which tries to keep tests from coupling to unnecessary system components like the GUI. Why are these rules being ignored? It seems reasonable to me that people are ignoring the rules of TDD because they don’t think they are doing TDD. They’re doing BDD instead, and the rules for BDD are “different”.

    No, they’re not. If you are doing BDD, you are doing TDD, and all the rules of good test design, and decoupling still apply. You don’t get a special license to create highly coupled tests just because you are doing BDD.

    “Robert C. Martin”

  5. Bob,

    I agree 100% with “You don’t get a special license to create highly coupled tests just because you are doing BDD.”

    I think, however, that there’s some confusion about the scope of TDD. http://xprogramming.com/images/circles.jpg shows TDD in the center circle, and Customer tests in the outer circle. The middle circle comprises practices that are developer-centric, like pair programming and refactoring. I think that a fair interpretation of this is that TDD is a developer-centric practice, and Customer tests serve a wider audience.

    You seem to be lumping unit and functional testing together under the banner of TDD, whereas a common interpretation would group unit testing with TDD, functional testing with Customer tests, and the two together under XP. If you accept that (note the “if” ;) ), then BDD is operating on the same level as XP, a level up from TDD. I think that’s where Dan is coming from here, at least in part.

  6. David, it’s true that Ron’s “Circle of Life” diagram shows TDD as a developer practice; but the circle of life is just one perspective on the practices of XP.

    1. Bob – I get that – that’s why I talked about interpretations vs absolute truths. In my experience, however, the most common interpretation is that TDD is about unit testing with a very tight red-green-refactor cycle, and other practices like Customer tests (functional tests, acceptance tests, etc, etc, etc) are about something more broad.

      I recall a conversation w/ Dan some years back about JBehave. Part of the idea was to simplify a very confusing testing landscape (ask 10 ppl the diff between acceptance, functional and integration tests and you’ll get 10 diff answers) by having one kind of test (scenario) that could model users interacting with a system and parts of a system interacting with other parts of the system with the same language. Doesn’t look like that particular goal has been achieved, and it’s unlikely that it ever will be because we can’t get consensus on definitions.

  7. Hi Bob,

    A lot of the teams I know who do BDD don’t automate the tests at all; they just use the scenario structure to explore the behaviour of the application. It’s more about the conversations than automation. I’ve seen teams gain huge traction just by writing a few key scenarios down, without tying them directly to code at all.

    I think the teams abusing BDD are seeing problems because they believe you *have* to automate scenarios, and because they’re confusing genuine regression testing practices with exploratory ones. I’d rather the focused on the quality of the conversations they’re having, rather than “good test design”, as this leads to the discovery of far more interesting scenarios, phrased in the language of the business, and which tend to lead themselves to easier maintainability anyway.

    “Good test design” is not IMO as valuable as understanding how to have useful, directed conversations, where the focus is firmly on the business. Everything you’ve covered in your comment focuses on the developer side of BDD, and suggests you’re not aware of the other aspects.

    These aspects include Feature Injection, Deliberate Discovery and Real Options; context questioning and outcome questioning; practices which I’ve taught to help teams understand the risk in their projects and address them early without a single test being written – unit or otherwise. Conversations around examples of how the system should behave are a key part of that risk discovery. While you might say “TDD helps with those conversations too”, we’ve developed specific practices *just in the domain of those conversations*, and while that might be the intent of TDD and entirely possible to do while practicing TDD, it’s not a focus that I’ve seen.

    I should probably write a blog about that. Or a book… *hint hint Dan*

  8. [...] Sure, if you’re a developer, and you only get to see the stuff that happens in development. Even then, it may be worth talking through not just the scenarios, but the larger-scale picture; discovering uncertainty and ignorance, paying to keep options open, and working towards a clear and compelling vision to deliver software that’s not only well-tested and functional, but which really, truly matters. Posted by liz at 1:08 pm [...]

  9. Dinesh Sharma · ·

    Spot on Liz. The value that BDD add is to bridge the communication gap between Business and Development team. BDD ensures that business and development team start complementing each other. It’s simplifies the old concept of what before how. IMHO, BDD solves what and TDD solves how issues.

  10. I understand that writing scenarios can help you think through your requirements. We used to call this proces “analysis”. The old saw was: “Don’t write a functional spec, write the user guide.”

    The reason I wrote FitNesse, so long ago, is that I saw Ward use Fit to automate the HP-35 user guide. He made me believe that there was a way to turn a document that was similar to a use guide into an executable specification. Of course we found that creating such specifications is hard, and that they often became documents that were so technical that only programmers could read them.

    Dan’s introduction of BDD was great, because it gave us a way to write these specifications in a way that kept them easy enough to read and write; and yet precise enough to automate. Cudos to Dan.

    What I find disturbing about the current thread is the apparent de-emphasis on automation. I shudder to think that organizations are creating scenarios and are then avoiding the obvious step of automating them. I can understand not automating scenarios that are still in flux; but please, please, automate them before programmers develop them. The organization that does not practice test automation is in for a nasty surprise when the CFO eventually tell them that the army of testers they use every release has gotten too expensive.

    It seems to me that if we lose the intent to automate, then we will also lose the precisions necessary to automate. And if we lose that precision, we lose the true value of the discipline. Attempting to analyze the behavior of a system using ambiguous language is a place we’ve already been, and don’t want to return to. Without the automation imperative, I fear that return may be imminent.

    At this point, with respect to the definition of BDD, I’m lost. I no longer know what BDD is. I thought it was a rather elegant and specific way to word precise statements of behavior with the intent to create executable specifications. But now the proponents of BDD are telling me that it’s transcended the writing of mere executable specifications, and has evolved into into a discipline of un-automated systems analysis. If that’s so, I wonder what the final D now stands for.

    1. Hi Bob,

      Thanks for both of your considered responses. I appreciate your kind words and the fact you are taking the time to respond in detail.

      The final D in BDD stands for “Development”: I’m not interested in behaviour-driven testing or behaviour-driven test automation (or even behaviour-driven design). I see the purpose of end-to-end software development as providing a capability. If it doesn’t do that it fails, however well tested or crafted it might be. The less software I can do this with, and the more targeted I can make that software, the better. Less software means less maintenance overhead and less to be ignorant about. (It’s surprising how quickly a codebase can accrete dark corners and areas of ignorance.)

      You’ve inspired me to follow up with another article about how I use risk-based, tactical TDD, because it’s still very much in my tool belt. Briefly though, you seem to be focusing on automation of executable specification as the goal, which implies all code is equally worthy of having automated tests and all behaviour needs to be equally well prescribed. I see test automation purely as a means to an end, namely to manage the risk of screwing things up when I change code. My understanding of that risk – and the way I engage with it – seems to be different from yours, and that is the subject of my follow-up. I view all software as a liability, on numerous levels, so well-written software is just less of a liability: it’s likely I’ll be less at risk when I have to change it. Automated tests may or may not help here, but I tend to assume they have no intrinsic value until I have evidence to the contrary. (Again, I’ll expand on this later.)

      BDD as it is now – and it probably makes more sense to compare it to the full XP stack than just the TDD discipline – is about delivering a capability. If we need software for that, so be it. We’ll develop the best software we can (or rather, the most appropriate in terms of risk and benefit – I’m fine with hacking too). We’ll try to flush out dragons early on to avoid surprises later. We’ll tackle the highest risk or least understood areas first. Maybe by exploring the business need we’ll discover a simpler way to provide the capability that doesn’t even involve software, or realise there were other options we hadn’t explored, or discover the original premise is flawed.

      The thing is, programming is only part of the ecosystem of providing capability to stakeholders, and BDD is about the latter which means it’s necessarily a bigger piece than the former.

      By the way, I want to get a teeshirt made that says “You don’t get a special license to create highly coupled tests just because you are doing BDD.”

    2. Hi Bob, I still like to automate. I’ve just found that there’s more value in having the conversations than in writing them down, and more value in writing them down than automating them. Often teams I’m coaching simply don’t have the expertise to automate, even if the language or technology they’re using allows it. In those situations, just starting with conversations makes a huge difference.

      Also, if we put the emphasis on doing the new and differentiating stuff first, the scenarios are usually in flux to start with; everything is essentially a spike that we stabilize later.

      A point on which I think we agree is that automating through the UI isn’t the *only* way to do this. I’ve seen some teams get traction with it, so I know it can be done usefully, but they usually had a large suite of integration tests too. For teams going through the UI, working out *which* scenarios are the right ones to automate can be tricky.

      In the last project I worked on, the frameworks on which we were building were so decoupled that it was nigh on impossible to create regression bugs, so there was no point. Good code design is IMO even more important than automated scenarios.

      I also believe we want the same thing: high-quality, valuable software. I’ve been able to steer people away from the Mess Driven Development you talk about by getting them to focus on the quality of the conversations they’re having. It turns out that once the steps are phrased in the right language, the underlying automation design has a tendency to fall out more cleanly.

  11. For a set of practices or a methodology that focuses on communication, BDD has failed to communicate its meaning .. specially if compared to XP or TDD … it’s very easy to piggyback on something well established and understood like XP and then say yeah … it’s like that but with communication … it’s also easy to say yeah whatever Gojko said on his Spec by Example book … but it is wrong and confusing

    I can look at my crystal ball and say that BDD is going to intersect next with Lean Startup … because really, that’s the most edgy stuff right now from the perspective of figuring out what software we need to write … and the communication part is gonna outgrow the business and include practices that relate to the customer … so BDD is like lean startup + XP

    All and all if this keeps growing, BDD is software development done right (to the best of our current knowledge) … which implies that the definition is a moving target as we keep finding better techniques … that doesn’t make sense, a single term cannot keep absorbing everything because it turns meaningless, and confusing, and energy gets lost in the arguments

    So my challenge to the BDD gurus is to write a book … and freeze BDD 1.0 or 3.0 ‘cos by my count we r in the 3rd generation

    1. Version 3 is probably about right. Initially I was trying to teach TDD and found I kept getting bogged down by irrelevant conversations about testing. So I changed the words. That was version 1. This is mid-2003.

      Chris Matts was working with me as a business analyst at ThoughtWorks – this is early 2004 – and made the connection between how I was describing the-thing-Kent-calls-TDD and how he liked to structure conversations with his stakeholders. That’s when the given-when-then vocabulary appeared. That was version 2.

      At this point – 2005 to about 2009 – I was describing BDD as an outside-in, multiple layer, multiple stakeholder methodology (as distinct from XP with its single on-site customer). There was confusion around my use of “outside-in”, as opposed to top-down or bottom-up. The best way I could describe outside-in was in terms of the Lean concept of “pull”: features needed a downstream stakeholder or they didn’t happen. Rachel Davies made that concrete for me. She said she doesn’t have deliverables on her projects, she has “consumables”: everything she produces has to have an identified consumer. Chris articulated this pull model really nicely as Feature Injection and then he started weaving Real Options into the mix to make the uncertainty in the process more explicit (rather than the false sense of certainty that comes from established agile planning and estimation techniques). That’s when I wrote The perils of estimation, when I realised how badly I’d been misleading people when I was coaching agile planning. That seems different enough to be version 3.

      Then Gojko captured my intent of BDD (v3) so well in his Specification by Example book that I didn’t think I had anything more to add in terms of a separate “BDD” book. That and the fact I seem to lack the discipline to produce a book. In my mind BDD really is software development done right. Why would I advocate anything I didn’t think was that? XP is also software development done right. It just presupposes some fortunate circumstances (great programmers, a single, embedded, articulate customer) that I didn’t have on my own projects. Plus I think Kent took liberties with the word “test” :)

      1. A few weeks back I was part of a discussion labeled “The difference between TDD and BDD”. From my point of view back then BDD is the combination of TDD with DDD with outside-in development, and forming a ubiquitous language. That picture has not changed, but I found the discussion hilarious enough from that point of reference.

        BDD tries to formulate a ubiquitous language for the product, yet fails to come up with a ubiquitous language for itself. I think this is the irony behind BDD right now. Sure, there are books, there are more books upcoming, yet, there already has been a critical mass claiming to do BDD, but not doing BDD at all. Just because it’s the hippest thing does not mean that you are doing it right.

        To me it’s only a matter of time when BDD will disappear due to this, if you keep the situation as is.

  12. There are a few aspect that i do not seem to understand. In the last reply you described BDD as being “software development done right”. The word “software” for me implies computer programing. I think that there is a semantic collision, at least for me, between this definition and the definition where BDD represents “delivering a capability for the stakeholder” and writing software for this being an optional part. The whole idea behind BDD emerged in the software development, your introduction to bdd article being a good example, you started answering some questions in the TDD process. I cannot say that BDD is not able to transcend the software development process, cannot argue that.
    Another point i fail to understand is about the “automation”. In your description about BDD at “Agile specifications, BDD and Testing eXchange” in November 2009 in London your definition about BDD included the words high-automation. In a previous response, you mentioned that automated tests may or may not help and you tend to assume that they bring no intrinsic value unless proven so. So what does the phrase high-automation stands for if not the automation of acceptance and unit tests?
    Bringing the ubiquitous language in all parts of the development process is an amazing asset which BDD i think emphasizes. What BDD has done for me at least was to cause a mind shift from implementation to specifying behavior.Reading your first article about BDD gives me the impression that your intent was the same. I think at least at the development team level BDD complies the Liskov substitution principle in regard to TDD.
    I would really love to see Nat Price or Steve Freeman’s opinion on this matter considering how they present all their TDD process, where everything starts from an acceptance test and then the unit tests are written inside this bigger loop.
    In the end, i must say that i highly appreciate the work done by all the persons involved in this conversation. Thank you.

    1. Hi Vadim,

      I describe BDD as high-automation, which it is, but automation isn’t the goal. I describe it as a software development methodology, but the production of software – for the sake of producing software – isn’t the goal either. The goal is delivering business capability quickly and sustainably (accepting that these are not necessarily compatible). It took me a few attempts to realise this. At the point where you’re describing the behaviour you want, you shouldn’t be thinking about implementation. If it helps, imagine an army of pixies called Thistle are going to do all the work. Many of those pixies will turn out to be software. Some of them won’t. Rachel Davies has a nice description of BDD that might provide more clarity. If nothing else it has some lovely drawings!

      With regard to test automation, I see value in automating scenarios or acceptance tests only when I think it will catch regressions that aren’t obvious. Automating happy paths isn’t that valuable to me – I’m in those all the time just by using the app. If I mess up some of the wiring the app usually won’t even start.

      Sometimes it is hard to automate acceptance tests, not because the app is poorly architected but because the technology stack doesn’t lend itself to that kind of automation. I’ve seen teams burn insane amounts of time trying to automate UI interactions, for instance, at huge cost and with almost no benefit. The opportunity cost, in terms of all the other things they could have done with that time, is considerable, and they’re usually doing it on someone else’s dime. I think there’s a duty of care involved in these kind of decisions. You shouldn’t automate “because we do” but because there is an identifiable benefit in the automation that outweighs its the cost in this case. Sometimes that investment is worth it, sometimes it isn’t, so it’s always worth asking the question.

      Nat and Steve have been using, applying and teaching TDD since the very early days of XP, and coaching their style of TDD was what inspired my original efforts with BDD. Nowadays people’s understanding of TDD in the wild has changed. They think of it mainly at the unit level as an incremental design technique. I would say TDD-as-intended – in the Nat and Steve (and Kent and Ward) style – is much more akin to BDD than TDD-as-usually-practised, modulo the constraints I described above.

  13. I might as well throw my 2 krónur in here, since there’s no reason to miss out on the fun.

    I have had differing opinions on this topic over the years. I wrote The World’s Shortest Article on BDD in 2006 at http://link.jbrains.ca/fJ3x8y in which I called BDD “TDD done correctly”. (Please ignore the arrogant language; I was young and foolish then.) Three years later I described a model at http://link.jbrains.ca/gj2svQ that I use to help people understand the transition from test-first programming to TDD to BDD to Continuous Delivery (later, to Lean Startup). Soon thereafter, I asked some UK-based BDD-oriented folks — you know who you are — about my characterisation of BDD as “using TDD to set up a pull system for features”. At the time, they claimed that that passed the Laugh Test. I wonder whether it still does. (Does it?)

    I’ll ignore, for a moment, the perils of arguing over labels, since we’ve done that since the day I met any of you.

    I take two key points from this exchange. (1) if I saw a group of only programmers doing BDD for and with themselves, I’d have a hard time distinguishing it from TDD; (2) while we might not need to automate all the scenarios, I think we gain a lot from writing them precisely enough to automate.

    I add one key point to this exchange. I do TDD in order to obtain feedback that helps me guide my designs, but I do BDD in order to obtain feedback that helps me avoid disappointing my customers. I teach it that way. I hope it helps.

    1. Hi JB,

      I’d pay more than 2 krónur for that. Your last two paragraphs were exactly the sentiment I was trying to get across. I particularly like the way you distinguish your motivation for using TDD and BDD. Thank you for that.

      1. Cheers, Dan. Please do pay me in Canadian dollars, however, since I’m leaving Iceland soon.

  14. BDD is like TDD if… you are producing unit tests, and like Fit if you are producing customer tests.

    There you go, in a sentence :)

    1. I have used Fit for unit tests and I have used JUnit for customer tests. Both have worked well.

      1. You aren’t going to force me to elaborate are you. :) Back in the day people use to talk about validation testing and verification testing. Remember that? That old chesnut called Software Engineering :) One was about testing/specifying requirements, the other about testing/specifying design.

        Anyway, just like Fit, BDD brings a lot to the table when it comes to specifying validation tests. The whole idea of capturing the conversation with the customer, BA, tester etc in an ubiquitous language that is readable by a broad audience and is also executable.

        You can also use BDD for verification tests as well, but in this regard it is pretty much the same as doing TDD with xUnit and agiledox (remember that too?)

        My main point is that there is not a lot new here. Not a problem, but the way things are being presented sorts of suggest there is, which is what is leading to the confusion I think.

        Paul.

  15. [...] Et la réponse de Dan North: BDD is like TDD if…. [...]

  16. [...] Dan North – BDD is like TDD if… [...]

  17. [...] BDD is like TDD if… — не надо путать BDD с TDD [...]

  18. [...] Using BDD to have a more holistic focus when doing TDD, which in turn promotes improved naming and test structure [...]

  19. [...] 过去几个月间,互联网上关于测试先行还是测试居后、代码测试比率或者行为驱动开发(BDD)是否真的是测试驱动开发(TDD)的讨论进行得如火如荼。InfoQ就此访问了行为驱动开发(BDD)和测试驱动开发领域知名的专家们,请他们对测试驱动开发(TDD),行为驱动开发(BDD)以及测试比率的运用发表各自的观点。(译注:此后测试驱动开发和行为驱动开发均用简称TDD和BDD代替。) [...]

Follow

Get every new post delivered to your Inbox.

Join 465 other followers

%d bloggers like this: