Introduction to Cucumber and BDD with examples
Cucumber examples can be found in selenium-samples-java/cucumber-parallel GitHub repository.
Before going into details how to create tests with Cucumber first is good to do some context introduction why this framework is created and getting more popular.
Test-driven development (TDD)
TDD (test driven development) is software development process in which developers first write the unit tests for feature or module based on requirements and then implement the feature or module itself. In short TDD cycle is "red > green > refactor":- Red - phase where tests are implemented according to requirements, but they still fail
- Green - phase where module or feature is implemented and tests pass
- Refactor - phase where a working code is made more readable and well structured
Behaviour-driven development (BDD)
BDD emerged from and extends TDD. Instead of writing unit tests from specification why not make the specification a test itself. The main idea is that business analysts, project managers, users or anyone without technical, but with sufficient business, knowledge can define tests.Gherkin
BDD ideas sound very nice but actually are not so easy to put in practice. In order to make documentation sufficient for testing, it should follow some rules. This is where Gherkin comes in place. It is a Business Readable, Domain Specific Language created especially to describe behavior without defining how to implement it. Gherkin serves two purposes: it is your project’s documentation and automated tests.Cucumber
Cucumber is not a testing tool it is a BDD tool for collaboration between all members of the team. So if you are using Cucumber just for automated testing you can do better. Testwise Cucumber is a framework that understands Gherkin and runs the automated tests. It sounds like a fairy tale, you get your documentation described in Gherkin and tests just run. Actually, it is not that simple, each step from documentation should have underlying test code that manipulates the application and should have test conditions. All process will be described in details with code samples below.Setup Maven project
The proper way to do things is by using some build automation tool. Most used ones are Gradle and Maven. Maven is more simple to use and works fine for normal workflows. Gradle can do anything you want to do, but it comes at a price of complexity since you have to write Groovy code for custom stuff. In order to use Cucumber-JVM (Cucumber implementation for the most popular JVM languages: Java, Groovy, Scala, etc.) in Java 8 POM looks like this:<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<cucumber.version>1.2.4</cucumber.version>
<selenium.version>2.48.2</selenium.version>
</properties>
<dependencies>
<dependency>
<groupId>info.cukes</groupId>
<artifactId>cucumber-java8</artifactId>
<version>${cucumber.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>info.cukes</groupId>
<artifactId>cucumber-junit</artifactId>
<version>${cucumber.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-server</artifactId>
<version>${selenium.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
In order to build project in Java 8 this should be specified explicitly in POM by using maven-compiler-plugin:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.3</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
Once the project is setup then real Cucumber usage can start.
Feature files
Everything starts with a plain text file with .feature extension written in Gherkin language that describes only one product feature. It may contain from one to many scenarios. Below is a file with a simple scenario for searching in Wikipedia.Feature: search Wikipedia
Scenario: direct search article
Given Enter search term 'Cucumber'
When Do search
Then Single result is shown for 'Cucumber'
In Gherkin, each line that isn’t blank has to start with a Gherkin keyword, followed by any text you like. The main keywords are:
- Feature
- Scenario
- Given, When, Then, And, But (Steps)
- Background
- Scenario Outline
- Examples
Don't repeat yourself
In some features, there might be one and the same Given steps before each scenario. In order to avoid copy/paste, it is better to define those steps as feature prerequisite with Background keyword.Feature: search Wikipedia
Background:
Given Open http://en.wikipedia.org
And Do login
Scenario: direct search article
Given Enter search term 'Cucumber'
When Do search
Then Single result is shown for 'Cucumber'
Data-driven testing
Cucumber makes it very easy to handle cases of different business scenarios with different input data and different results based on that input data. The scenario is defined with Scenario Outline. Then data is fed to this scenario with Examples table where variables are defined with concrete values. The example below shows a scenario where a search is done for two keywords and expected results for each is defined. It is very easy just to add more keywords and expected result which is actually treated by Cucumber reporting as a different scenario.Feature:
Scenario Outline:
Given Enter search term '<searchTerm>'
When Do search
Then Multiple results are shown for '<result>'
Examples:
| searchTerm | result |
| mercury | Mercury may refer to: |
| max | Max may refer to: |
Organise features and scenarios
Tags can be used in order to group feature files. A tag can be applied as annotation above Feature or Scenario keyword in .feature file. Having correctly tagged different scenarios or features runners can be created per each tag and run when needed. See more for tags in Cucumber tags page.Runners
Next step in the process is to add runners. Cucumber supports running tests with JUnit and TestNG. In the current post, JUnit will be used. It has been imported in POM project file with cucumber-junit. In order to run a test with JUnit a special runner class should be created. The very basic form of the file is an empty class with @RunWith(Cucumber.class) annotation.import org.junit.runner.RunWith;
import cucumber.api.junit.Cucumber;
@RunWith(Cucumber.class)
public class RunWikipediaTest {
}
This will run the tests and JUnit results file will be generated. Although it works it is much better to provide some Cucumber options with @CucumberOptions annotation.
import org.junit.runner.RunWith;
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;
@RunWith(Cucumber.class)
@CucumberOptions(
format = {
"json:target/cucumber/wikipedia.json",
"html:target/cucumber/wikipedia.html",
"pretty"
},
tags = {"~@ignored"}
)
public class RunWikipediaTest {
}
Runner above runs all feature files which does not have tag @ignored and outputs the results in three different formats: JSON in target/cucumber/wikipedia.json file, HTML in target/cucumber/wikipedia.html and Pretty - a console output with colours. All formatting options are available in Cucumber options page.
Runners strategy
Different runners can be created for different purposes. There could be runners based on specific tags and those to be invoked in specific cases like regression and smoke testing, or full or partial regression testing. Depends on the needs. It is possible to create runners for each feature and run tests in parallel. There is a way to automatically generate runners and run tests in parallel. This will speed up execution and will keep the project clean from unessential runners. More details can be found in Running Cucumber tests in parallel post.Runners feature file
If the runner is not linked to any feature file (no features section defined in @CucumberOptions, like the one shown above) then runner automatically searches for all feature files in current and all child folders.Java package structure is represented as a folder structure. Feature files can be placed in some package structure in resources folder of the project, e.g. com.automationrhapsody.cucumber.parallel.tests.wikipedia. When compiled those are copied to same folder structure in the output. So in order to run all feature files from package above runner class need to be placed in the same package in java folder of the project. It is possible to place the runner in parent package and it will still work as it searches all child folders e.g com.automationrhapsody.cucumber.parallel.tests will work. Placing in a different package will not work though, e.g. com.automationrhapsody.cucumber.parallel.tests.other will not work. There will be a message:
None of the features at [classpath:com/automationrhapsody/cucumber/parallel/tests/other] matched the filters: [~@ignored]
This means either there are no feature files into com.automationrhapsody.cucumber.parallel.tests.other resources package or those files have tag @ignored.
The easiest way is to put just one runner into default package and it will run all feature files available. This gives more limitations than benefits in the long run though.
Running feature files without step definitions
So far feature file and the runner has been created. If now tests are started build will successes but tests will be skipped. This is because there is no implementation available for Given, When, Then commands in the feature file. Those should be implemented and Cucumber gives hints in build output how to do it.You can implement missing steps with the snippets below:
Given("^Enter search term 'Cucumber'$", () -> {
// Write code here that turns the phrase above into concrete actions
throw new PendingException();
});
Step definitions code / glue
So far feature file has been defined with a runner for it. In terms of BDD this is OK, but in terms of testing a step, definitions should be created so tests can actually be executed. As shown in hint above a method with annotation @Given is needed. Annotation text is actually a regular expression this is why it is good to start with ^ and end with $ which means the whole line from the feature file to be matched. In order to make the step reusable, some regular expression can be added in order to be able to pass different search terms, not only 'Cucumber'. If there is regular expression defined then the method can take arguments which will be actually what regex will match in feature file text. Step definition for Given command is:@Given("^Enter search term '(.*?)'$")
public void searchFor(String searchTerm) {
WebElement searchField = driver.findElement(By.id("searchInput"));
searchField.sendKeys(searchTerm);
}
(.*?) is very basic regex, but it can do the work. It matches all zero or more characters surrounded by quotes in the end of the line after “Enter search term ” text. Once matched this is passed as argument to searchFor() method invocation.
Full step definitions for initial feature file are shown in class below:
package com.automationrhapsody.cucumber.parallel.tests.wikipedia;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import cucumber.api.java.After;
import cucumber.api.java.Before;
import cucumber.api.java.en.Given;
import cucumber.api.java.en.Then;
import cucumber.api.java.en.When;
import static junit.framework.Assert.assertTrue;
import static junit.framework.TestCase.assertFalse;
import static org.junit.Assert.assertEquals;
public class WikipediaSteps {
private WebDriver driver;
@Before
public void before() {
driver = new FirefoxDriver();
driver.navigate().to("http://en.wikipedia.org");
}
@After
public void after() {
driver.quit();
}
@Given("^Enter search term '(.*?)'$")
public void searchFor(String searchTerm) {
WebElement searchField = driver.findElement(By.id("searchInput"));
searchField.sendKeys(searchTerm);
}
@When("^Do search$")
public void clickSearchButton() {
WebElement searchButton = driver.findElement(By.id("searchButton"));
searchButton.click();
}
@Then("^Single result is shown for '(.*?)'$")
public void assertSingleResult(String searchResult) {
WebElement results = driver
.findElement(By.cssSelector("div#mw-content-text.mw-content-ltr p"));
assertFalse(results.getText().contains(searchResult + " may refer to:"));
assertTrue(results.getText().startsWith(searchResult));
}
}