Introduction to Cucumber and BDD with examples

Last Updated on by

Post summary: Code examples and introduction to Cucumber, a framework that runs automated tests written in behavior driven development (BDD) style.

Cucumber examples can be found in selenium-samples-java/cucumber-parallel GitHub repository.

Before going into details how to create tests with Cucumber first is good to do some context introduction why this framework is created and getting more popular.

Test-driven development (TDD)

TDD (test driven development) is software development process in which developers first write the unit tests for feature or module based on requirements and then implement the feature or module itself. In short TDD cycle is “red > green > refactor”:

  • Red – phase where tests are implemented according to requirements, but they still fail
  • Green – phase where module or feature is implemented and tests pass
  • Refactor – phase where a working code is made more readable and well structured

Behaviour-driven development (BDD)

BDD emerged from and extends TDD. Instead of writing unit tests from specification why not make the specification a test itself. The main idea is that business analysts, project managers, users or anyone without technical, but with sufficient business, knowledge can define tests.

Gherkin

BDD ideas sound very nice but actually are not so easy to put in practice. In order to make documentation sufficient for testing, it should follow some rules. This is where Gherkin comes in place. It is a Business Readable, Domain Specific Language created especially to describe behavior without defining how to implement it. Gherkin serves two purposes: it is your project’s documentation and automated tests.

Cucumber

Cucumber is not a testing tool it is a BDD tool for collaboration between all members of the team. So if you are using Cucumber just for automated testing you can do better. Testwise Cucumber is a framework that understands Gherkin and runs the automated tests. It sounds like a fairy tale, you get your documentation described in Gherkin and tests just run. Actually, it is not that simple, each step from documentation should have underlying test code that manipulates the application and should have test conditions. All process will be described in details with code samples below.

Setup Maven project

The proper way to do things is by using some build automation tool. Most used ones are Gradle and Maven. Maven is more simple to use and works fine for normal workflows. Gradle can do anything you want to do, but it comes at a price of complexity since you have to write Groovy code for custom stuff. In order to use Cucumber-JVM (Cucumber implementation for the most popular JVM languages: Java, Groovy, Scala, etc.) in Java 8 POM looks like this:

<properties>
	<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
	<cucumber.version>1.2.4</cucumber.version>
	<selenium.version>2.48.2</selenium.version>
</properties>
<dependencies>
	<dependency>
		<groupId>info.cukes</groupId>
		<artifactId>cucumber-java8</artifactId>
		<version>${cucumber.version}</version>
		<scope>test</scope>
	</dependency>
	<dependency>
		<groupId>info.cukes</groupId>
		<artifactId>cucumber-junit</artifactId>
		<version>${cucumber.version}</version>
		<scope>test</scope>
	</dependency>
	<dependency>
		<groupId>org.seleniumhq.selenium</groupId>
		<artifactId>selenium-server</artifactId>
		<version>${selenium.version}</version>
		<scope>test</scope>
	</dependency>
</dependencies>

In order to build project in Java 8 this should be specified explicitly in POM by using maven-compiler-plugin:

<build>
	<plugins>
		<plugin>
			<groupId>org.apache.maven.plugins</groupId>
			<artifactId>maven-compiler-plugin</artifactId>
			<version>3.3</version>
			<configuration>
				<source>1.8</source>
				<target>1.8</target>
			</configuration>
		</plugin>
	</plugins>
</build>

Once the project is setup then real Cucumber usage can start.

Feature files

Everything starts with a plain text file with .feature extension written in Gherkin language that describes only one product feature. It may contain from one to many scenarios. Below is a file with a simple scenario for searching in Wikipedia.

Feature: search Wikipedia

  Scenario: direct search article
    Given Enter search term 'Cucumber'
    When Do search
    Then Single result is shown for 'Cucumber'

In Gherkin, each line that isn’t blank has to start with a Gherkin keyword, followed by any text you like. The main keywords are:

  • Feature
  • Scenario
  • Given, When, Then, And, But (Steps)
  • Background
  • Scenario Outline
  • Examples

More information can be found in Cucumber reference page.

Don’t repeat yourself

In some features, there might be one and the same Given steps before each scenario. In order to avoid copy/paste, it is better to define those steps as feature prerequisite with Background keyword.

Feature: search Wikipedia

  Background:
    Given Open http://en.wikipedia.org
    And Do login

  Scenario: direct search article
    Given Enter search term 'Cucumber'
    When Do search
    Then Single result is shown for 'Cucumber'

Data-driven testing

Cucumber makes it very easy to handle cases of different business scenarios with different input data and different results based on that input data. The scenario is defined with Scenario Outline. Then data is fed to this scenario with Examples table where variables are defined with concrete values. The example below shows a scenario where a search is done for two keywords and expected results for each is defined. It is very easy just to add more keywords and expected result which is actually treated by Cucumber reporting as a different scenario.

Feature:

  Scenario Outline:
    Given Enter search term '<searchTerm>'
    When Do search
    Then Multiple results are shown for '<result>'

    Examples:
      | searchTerm | result                |
      | mercury    | Mercury may refer to: |
      | max        | Max may refer to:     |

Organise features and scenarios

Tags can be used in order to group feature files. A tag can be applied as annotation above Feature or Scenario keyword in .feature file. Having correctly tagged different scenarios or features runners can be created per each tag and run when needed. See more for tags in Cucumber tags page.

Runners

Next step in the process is to add runners. Cucumber supports running tests with JUnit and TestNG. In the current post, JUnit will be used. It has been imported in POM project file with cucumber-junit. In order to run a test with JUnit a special runner class should be created. The very basic form of the file is an empty class with @RunWith(Cucumber.class) annotation.

import org.junit.runner.RunWith;

import cucumber.api.junit.Cucumber;

@RunWith(Cucumber.class)
public class RunWikipediaTest {
}

This will run the tests and JUnit results file will be generated. Although it works it is much better to provide some Cucumber options with @CucumberOptions annotation.

import org.junit.runner.RunWith;

import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;

@RunWith(Cucumber.class)
@CucumberOptions(
		format = {
				"json:target/cucumber/wikipedia.json",
				"html:target/cucumber/wikipedia.html",
				"pretty"
		},
		tags = {"~@ignored"}
)
public class RunWikipediaTest {
}

Runner above runs all feature files which does not have tag @ignored and outputs the results in three different formats: JSON in target/cucumber/wikipedia.json file, HTML in target/cucumber/wikipedia.html and Pretty – a console output with colours. All formatting options are available in Cucumber options page.

Runners strategy

Different runners can be created for different purposes. There could be runners based on specific tags and those to be invoked in specific cases like regression and smoke testing, or full or partial regression testing. Depends on the needs. It is possible to create runners for each feature and run tests in parallel. There is a way to automatically generate runners and run tests in parallel. This will speed up execution and will keep the project clean from unessential runners. More details can be found in Running Cucumber tests in parallel post.

Runners feature file

If the runner is not linked to any feature file (no features section defined in @CucumberOptions, like the one shown above) then runner automatically searches for all feature files in current and all child folders.

Java package structure is represented as a folder structure. Feature files can be placed in some package structure in resources folder of the project, e.g. com.automationrhapsody.cucumber.parallel.tests.wikipedia. When compiled those are copied to same folder structure in the output. So in order to run all feature files from package above runner class need to be placed in the same package in java folder of the project. It is possible to place the runner in parent package and it will still work as it searches all child folders e.g com.automationrhapsody.cucumber.parallel.tests will work. Placing in a different package will not work though, e.g. com.automationrhapsody.cucumber.parallel.tests.other will not work. There will be a message:

None of the features at [classpath:com/automationrhapsody/cucumber/parallel/tests/other] matched the filters: [~@ignored]

This means either there are no feature files into com.automationrhapsody.cucumber.parallel.tests.other resources package or those files have tag @ignored.

The easiest way is to put just one runner into default package and it will run all feature files available. This gives more limitations than benefits in the long run though.

Running feature files without step definitions

So far feature file and the runner has been created. If now tests are started build will successes but tests will be skipped. This is because there is no implementation available for Given, When, Then commands in the feature file. Those should be implemented and Cucumber gives hints in build output how to do it.

You can implement missing steps with the snippets below:

Given("^Enter search term 'Cucumber'$", () -> {
	// Write code here that turns the phrase above into concrete actions
	throw new PendingException();
});

Step definitions code / glue

So far feature file has been defined with a runner for it. In terms of BDD this is OK, but in terms of testing a step, definitions should be created so tests can actually be executed. As shown in hint above a method with annotation @Given is needed. Annotation text is actually a regular expression this is why it is good to start with ^ and end with $ which means the whole line from the feature file to be matched. In order to make the step reusable, some regular expression can be added in order to be able to pass different search terms, not only ‘Cucumber’. If there is regular expression defined then the method can take arguments which will be actually what regex will match in feature file text. Step definition for Given command is:

@Given("^Enter search term '(.*?)'$")
public void searchFor(String searchTerm) {
	WebElement searchField = driver.findElement(By.id("searchInput"));
	searchField.sendKeys(searchTerm);
}

(.*?) is very basic regex, but it can do the work. It matches all zero or more characters surrounded by quotes in the end of the line after “Enter search term ” text. Once matched this is passed as argument to searchFor() method invocation.

Full step definitions for initial feature file are shown in class below:

package com.automationrhapsody.cucumber.parallel.tests.wikipedia;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;

import cucumber.api.java.After;
import cucumber.api.java.Before;
import cucumber.api.java.en.Given;
import cucumber.api.java.en.Then;
import cucumber.api.java.en.When;

import static junit.framework.Assert.assertTrue;
import static junit.framework.TestCase.assertFalse;
import static org.junit.Assert.assertEquals;

public class WikipediaSteps {

	private WebDriver driver;

	@Before
	public void before() {
		driver = new FirefoxDriver();
		driver.navigate().to("http://en.wikipedia.org");
	}

	@After
	public void after() {
		driver.quit();
	}

	@Given("^Enter search term '(.*?)'$")
	public void searchFor(String searchTerm) {
		WebElement searchField = driver.findElement(By.id("searchInput"));
		searchField.sendKeys(searchTerm);
	}

	@When("^Do search$")
	public void clickSearchButton() {
		WebElement searchButton = driver.findElement(By.id("searchButton"));
		searchButton.click();
	}

	@Then("^Single result is shown for '(.*?)'$")
	public void assertSingleResult(String searchResult) {
		WebElement results = driver
			.findElement(By.cssSelector("div#mw-content-text.mw-content-ltr p"));
		assertFalse(results.getText().contains(searchResult + " may refer to:"));
		assertTrue(results.getText().startsWith(searchResult));
	}
}

Before and After

As noticeable above there is method annotated with @Before. This is executed before each scenario being run. It is used to allocate resources needed for a particular scenario to run. It is a good idea to instantiate WebDriver into this method. @After method is run at the end of the scenario so this is where resources are released, e.g. quitting WebDriver.

Page objects

Steps definitions code above is NOT well written. It is not a good idea to locate elements directly in the method where they are used. Never do this in real automation! Always use Page Object pattern. More details on it can be found in Page objects design pattern post.

Conclusion

BDD as an idea is good as it is according to Agile methodologies and stories definitions. Cucumber is very good BDD tool giving lots of flexibility and features with a big community supporting it. Cucumber-JVM is very easy to start working with. Be careful though, when steps definitions grow in count the main challenge will be to make those reusable. Challenge will be to avoid writing new steps code if such is already written for semantically the same commands but written in different words.

Related Posts

Category: Java, Tutorials | Tags: