What are the differences between BDD frameworks for Java? [closed]

Answer 1

My team have been using JBehave for some time. It uses plain text files to store specifications. Every step (Given, When, Then) is then executed by a certain method which can extract parameters from the step. Scenarios can be indented and well formatted which helps a lot if clients want to verify them.

There are some problems, too. We have switched to Java 6. Sometimes some scenario steps are ignored during execution. It may cause a lot of trouble figuring out where's the bug.

Answer 2

My team have used JBehave with success - we moved to it after using EasyB and found the plain text scenario files easier to deal with.

Answer 3

I originally did my BDD with plain jUnit but I've been looking at JDave lately because it's almost 1:1 to what I was doing with jUnit. It also runs on top of jUnit so it already works on Eclipse and is also easy to configure to work on continuous integration systems such as Hudson. Can't really compare it with others but my experiences with JDave have been good so far.

Oh and it's never a stupid idea to use mocks! They're not tied to TDD/BDD specifically, their purpose is to ease the burden of testing in general.

Answer 4

I've just finished comparing three BDD frameworks for Java. Obviously my findings have a fairly short use-by date.

Concordion

• Very flexible
• Very pretty report output
• Nice plugin framework
• Poorly documented. I had to read the source to figure it out (luckily its extremely good quality).
• Fixtures seemed likely to end up tightly coupled to the html.

EasyB

• Very shallow learning curve (even for non-Groovy Developers)
• Extremely powerful DBUnit integration
• Apparently no support for parameters (leads to either very vague stories or duplication between text and code (edit: actually there is but the documentation for it was very well hidden.)
• Story and Code are very tightly coupled (same file)
• Very basic report output
• Couldn't get IntelliJ plugin to work
• Inactive community (Maven plugin seems to have been broken for three months - not many code examples to draw on)

JBehave

• Extremely powerful and flexible (eg reduction of boiler-plate through composition of stories as pre-requisites)
• Extensive (if fragmented) documentation and examples
• Extensive (if overwhelming) support for different frameworks and environments
• Excellent separation of story files from code
• Looks to have a pretty active community and much more examples and discussion of it on web.
• Quite a steep learning curve (took me 3-4 times longer to figure out than Concordion/EasyB)

I didn't have the chance to try out Cuke4Duke of JDave as I would have liked, but will probably push for JBehave at this time.

Answer 5

Wow, I see the topic is hot, lot of good answers...

Irony aside, I recently discovered BDD and found the concept interesting. Hey, it forces to write both tests... and specifications! As surprising as it might seem, the latter can be also missing in some projects... Or just lacking the precision that BDD forces to introduce.

The Behavior Driven Development article summarizes the concept and links to some good articles (like the one written by Andrew Glover). Moreover, to the topic of this thread, it gives a rather comprehensive (I suppose) listing of BDD frameworks, a good number of them being for Java.
It doesn't solve the problem of choosing the framework but at least it will ease the search...

Since BDD relies heavily on readability of test code, I suppose a good criterion of choice is to look at the quick tours/tutorial and see which one seems the more fitting your style. Other criteria could be the fact a framework leverage tools you are familiar with (unit test, mocking), usage with IDE, and so on.

Answer 6

"pros and cons" might be different things for different people. I usually have a look at

• development activity, e.g. are new releases likely or is the last release 2 years old.
• maturity, e.g. how long has it been around, are there tutorials and maybe even books available. (I don't read these books, it's just a sign of adoption.)
• tool support, e.g. is there an Eclipse plugin, Ant support, etc
• size of the dependencies, I don't like frameworks that come with everything of their own. e.g. I want to chose my mocking framework myself.
• kind of license, this is important for me because of legal terms in the company I work for.
• compatibility with related tools, e.g. does it use Gherkin language or not.

And from some frameworks I had a look at

• Instinct bad: last activity Mar 2010, good: ASF license
• JDave bad: comes with matchers and mocks, good: last activity Jan 2011, ASF license
• easyb bad: last activity Oct 2010, not sure: it uses Groovy. This might be ok, but would be a problem for adoption in my case.
• beanspec bad: only one version in 2007, this is dead
• bdoc bad: last activity Jan 2010, not sure: it looks like going the other way, creating a report from the code.
• spock bad: maybe a bit extreme, this is a complete testing framework, not only BDD, good: very active, very cool.
• jbehave, the "mother" of all BDD in Java, bad: very powerful = complex, incompatible license (for me), comes with almost every test library and much more, good: based on RSpec and therefore compatible, eclipse plugins, maven integration, very active community
• ginkgo4j, a BDD framework for Java also based on Ruby's RSpec but using Java lambda's (instead of annotations) to allow you to create highly contextual, highly readable tests. Simple. Very powerful. Open source Apache 2 license.

Concerning the mocks: You definitely need a mocking framework as well. The BDD frameworks just help you in writing the specs, but some tests will need mocks or stubs, esp. when you design top down (from overview to detail).