Loading...

Follow John Ferguson Smart on Feedspot

Continue with Google
Continue with Facebook
or

Valid
John Ferguson Smart by Dolly Kumar - 1w ago
Lightweight Mentoring packages form
  • Name
    First Last
  • Email
  • Username
  • Password
    Enter Password Confirm Password
  • Address
    Street Address Address Line 2 City County / State / Region ZIP / Postal Code AfghanistanAlbaniaAlgeriaAmerican SamoaAndorraAngolaAntigua and BarbudaArgentinaArmeniaAustraliaAustriaAzerbaijanBahamasBahrainBangladeshBarbadosBelarusBelgiumBelizeBeninBermudaBhutanBoliviaBosnia and HerzegovinaBotswanaBrazilBruneiBulgariaBurkina FasoBurundiCambodiaCameroonCanadaCape VerdeCayman IslandsCentral African RepublicChadChileChinaColombiaComorosCongo, Democratic Republic of theCongo, Republic of theCosta RicaCôte d'IvoireCroatiaCubaCuraçaoCyprusCzech RepublicDenmarkDjiboutiDominicaDominican RepublicEast TimorEcuadorEgyptEl SalvadorEquatorial GuineaEritreaEstoniaEthiopiaFaroe IslandsFijiFinlandFranceFrench PolynesiaGabonGambiaGeorgiaGermanyGhanaGreeceGreenlandGrenadaGuamGuatemalaGuineaGuinea-BissauGuyanaHaitiHondurasHong KongHungaryIcelandIndiaIndonesiaIranIraqIrelandIsraelItalyJamaicaJapanJordanKazakhstanKenyaKiribatiNorth KoreaSouth KoreaKosovoKuwaitKyrgyzstanLaosLatviaLebanonLesothoLiberiaLibyaLiechtensteinLithuaniaLuxembourgMacedoniaMadagascarMalawiMalaysiaMaldivesMaliMaltaMarshall IslandsMauritaniaMauritiusMexicoMicronesiaMoldovaMonacoMongoliaMontenegroMoroccoMozambiqueMyanmarNamibiaNauruNepalNetherlandsNew ZealandNicaraguaNigerNigeriaNorthern Mariana IslandsNorwayOmanPakistanPalauPalestine, State ofPanamaPapua New GuineaParaguayPeruPhilippinesPolandPortugalPuerto RicoQatarRomaniaRussiaRwandaSaint Kitts and NevisSaint LuciaSaint Vincent and the GrenadinesSamoaSan MarinoSao Tome and PrincipeSaudi ArabiaSenegalSerbiaSeychellesSierra LeoneSingaporeSint MaartenSlovakiaSloveniaSolomon IslandsSomaliaSouth AfricaSpainSri LankaSudanSudan, SouthSurinameSwazilandSwedenSwitzerlandSyriaTaiwanTajikistanTanzaniaThailandTogoTongaTrinidad and TobagoTunisiaTurkeyTurkmenistanTuvaluUgandaUkraineUnited Arab EmiratesUnited KingdomUnited StatesUruguayUzbekistanVanuatuVatican CityVenezuelaVietnamVirgin Islands, BritishVirgin Islands, U.S.YemenZambiaZimbabwe Country
  • Phone
  • Credit Card
    American Express
    Discover
    MasterCard
    Visa
    Card Number Month010203040506070809101112 Year20192020202120222023202420252026202720282029203020312032203320342035203620372038 Expiration Date   Security Code Cardholder Name

The post test gravity form appeared first on John Ferguson Smart.

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Assertions are an important part of any test automation framework, and Serenity gives us many options. You can of course use standard JUnit, Hamcrest or AssertJ assertions at any point in a Screenplay test. But more recent versions of Serenity Screenplay provide an alternative approach, which many developers find easier to use and faster to write: the serenity-ensure module.

Introducing serenity-ensure

The Ensure class produces a Performable, so you can integrate them directly into the attemptsTo() method. It also has a very readable DSL and lets you use code completion to discover the assertions you can use for different values, making writing assertions easier and quicker.

An example of code equivalent to the above can be seen here. Suppose you have an input field on your HTML form to enter the age, which you have identified in your Screenplay test using a Target:

Target AGE_FIELD = Target.the("Age field").locatedBy("#age");

Using serenity-ensure, you could check the value of this field using the Ensure.that() method, like this:

...
sam.attemptsTo(
    Ensure.that(AGE).value().isEqualTo("40") 
);
Getting started with serenity-ensure assertions

To add serenity-ensure to your project,  you need to add a new dependency, as shown here for Maven:

<dependency>
 <groupId>net.serenity-bdd</groupId>
 <artifactId>serenity-ensure</artifactId>
 <version>${serenity.version}</version>
 <scope>test</scope>
 </dependency>

The starting point for all serenity-ensure assertions is the  net.serenitybdd.screenplay.ensure.Ensure class.

From here, you use the Ensure.that() method, passing the value you want to check. This can be an actual value (String, Integer, LocalDate etc), a Screenplay Question, or a web element locator. The actual assertions available can be different for different data types, and this is one of the things that makes serenity-ensure so convenient to use: Once you have entered  Ensure.that(someValue) in your IDE, you can use auto-complete to explore the range of available assertions for that type. There are over a hundred different assertion methods, covering web elements, Strings, Dates and Times, integer and floating point numbers, and collections. You can see the full set of assertions in the Serenity Documentation, but let’s take a look at some of the highlights.

Working with assertions about web elements

The serenity-ensure module has a rich range of assertions related to web elements. We have already seen some of these in the previous code. Other assertions include isDisplayed() and isEnabled(). We can use either a Target instance directly, or use the ElementLocated.by() method to specify a Target, a By locator, or a CSS or XPath expression. For example:

aster.attemptsTo(
    Ensure.that(ElementLocated.by("#firstName")).isDisplayed()
);

You can use methods like value(), text() or textContent() to read from a web element. Once you have obtained the value, you can use a range of String assertions, such as startsWith() shown here:

aster.attemptsTo(
    Ensure.that(FIRST_NAME).value().startsWith("Joe")
);
Working with assertions about collections

We often need to reason about collections of values, rather than just individual values. We can use methods like values() and textContentValues() to make assertions about collections of web elements. For example:

aster.attemptsTo(
    Ensure.that(ElementLocated.by("#colors option"))
          .values()
          .contains("red","blue","green") );

We can also make assertions about the elements themselves. TheMatchingElement class gives us a set of methods to make assertions about these elements, and methods such as allMatch(), anyMatch() and noneMatch() complete the picture:

aster.attemptsTo(
    Ensure.thatTheSetOf(ElementsLocated.by(".train-line"))
          .allMatch(TheMatchingElement.containsText("Line"))
);
Converting types

We can also use converter methods such as asAnInteger(), asADouble, or asADate() to convert the String value returned by WebDriver into a form more convenient to make assertions on. For example, we could refactor the code we saw earlier to reason about the age value as an integer like this:

...
sam.attemptsTo(
    Ensure.that(AGE).value().asAnInteger().isGreaterThan(18) 
);

Alternatively, if you wanted to make an assertion about a date or a time, you could write code like the following:

aster.attemptsTo(
    Ensure.that(ElementLocated.by("#currentDate"))
          .value()
          .asADate("dd-MM-yyyy")
          .isBefore(dateLimit)
);
Want to learn more?

This is just a very brief taster of what the serenity-ensure module can do. You can read the full documentation for the serenity-ensure module here. If you want to learn more about Serenity Screenplay, you can follow the introductory tutorial here. Or if you prefer video tutorials and guided exercises, the Serenity Dojo Screenplay Course will be getting a module on serenity-ensure later this week.

The post Serenity Ensure – Fluent Assertions in Serenity Screenplay appeared first on John Ferguson Smart.

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Join John Ferguson Smart in Madrid this June at Expo:QA and learn the finer points of Advanced BDD Test Automation practices

In this workshop with John Smart, author of BDD in Action, you will be introduced to the principles and practices of applying software engineering design practices to test automation, looking at concepts such as the Single Responsibility Principle, Separation of Concerns, and effective layering.

You will put these principles into practice yourself on a real web site, using action classes and lean page objects to write clean, readable, well-structured test automation code.

You will then discover the powerful Screenplay pattern, and see how it can make your test suites even more flexible and robust.

Objectives

In this very hands-on and practical session, John will show you powerful and applicable techniques to:

  • Write more automated tests faster
  • Write higher quality automated tests, making them faster, more reliable and easier to maintain
  • Increase confidence in your automated tests, and
  • Reduce the cost of maintaining your automated test suites
Outcomes

From this workshop, you will learn how to use advanced development skills to write more robust and more maintainable test and to write faster and more stable web tests with advanced WebDriver and Serenity BDD.

Target Audience

The workshop is designed to introduce Testers who are comfortable with basic Java and Selenium to more advanced and more sustainable test automation practices. You will use an existing test suite running against a real-world web site, where you will be able to clone from Github, before the workshop.

The workshop will work for any level, mainly for general and intermediate, but with extension activities for more advanced participants.

The post BDD in Action: Advanced BDD Test Automation appeared first on John Ferguson Smart.

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Gherkin Refactoring Patterns

Good Gherkin is easy to read but hard to write. And while there are many tips and tricks for writing good Gherkin, often teams still struggle to keep their scenarios clean, informative and readable.

In this talk, we will take a practical look at some real-world Gherkin scenarios, and see why they stink. More importantly, we present a set of simple refactoring patterns which can help you improve your own scenarios.

John Ferguson Smart (@wakaleo)

The post Cukenfest london – Open space and workshops appeared first on John Ferguson Smart.

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

John Ferguson Smart is an international speaker, consultant, author and trainer well known in the Agile community for his many books, articles and presentations, particularly in areas such as BDD, TDD, test automation, software craftsmanship and team collaboration. John is the author of the best-selling BDD in Action, as well as Jenkins: The Definitive Guide and Java Power Tools and also leads development on the innovative Serenity BDD test automation library. Based in London, John helps clients around the world deliver better software sooner through advanced agile practices such as BDD, test automation, software craftsmanship, and scaled agile patterns.

Presenting

Engage! Bringing teams together to deliver software that makes a difference

International speaker and author of “BDD in Action” John Ferguson Smart shows how you can multiply your team’s productivity and innovation by engaging the creativity of your whole team from the outset. Drawing from his long experience helping teams deliver better software faster and more effectively, John will discuss the latest practical techniques leveraged from Behaviour Driven Development, Lean Enterprise, DevOps, and Test Automation, combined with research in Psychology and Team Performance, to show you how to get the best out of your teams.

Learn about the new roles of business analysts, developers and testers in the future of software development, where testers can play a vital role in not only detecting defects but preventing them. Discover how you can make test automation happen during, not after, the sprint, and how to engage the creativity of the whole team right from the word “go”.

The post National Software Testing Conference appeared first on John Ferguson Smart.

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

It was the time of Da Vinci and Michelangelo. It was also the time of Machiavelli and the Medici. Artists working on timeless masterpieces crossed paths with mercenary captains, contracted to do a very specific job.

In this keynote talk, John Smart will address important questions with deep implications for any IT team, or any organisation trying to make a difference, or simply to get the most value out of their IT projects.

Who is your real customer? Is there a cost to quality? Are you building an artwork that will last, or simply fulfilling a contract?

An inspiring and entertaining talk that will take attendees on journey from the Italian Renaissance to Silicon Valley and the City of London, and see what lessons can be learned about cultures, attitudes and work ethics today.

The post Condottieri e Artisti – Mercenaries and Artists in the IT industry appeared first on John Ferguson Smart.

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

This tutorial show you how to get started with REST-API testing using Serenity and Cucumber 4.

Get the code

Git:

git clone https://github.com/serenity-bdd/serenity-rest-starter.git
cd serenity-rest-starter

Or simply download a zip file.

The starter project

The best place to start with Serenity and Cucumber is to clone or download the starter project on Github (https://github.com/serenity-bdd/serenity-rest-starter). This project gives you a basic project setup, along with some sample tests and supporting classes. The starter project comes bundled with a sample SpringBoot web service, and some RestAssured-based tests. The project also illustrates how you might use Freemarker to prepare test data for your scenarios.

The project directory structure

The project has build scripts for both Maven and Gradle, and follows the standard directory structure used in most Serenity projects:

src
  + main
  + test
    + java                                Test runners and supporting code
    + resources
      + features                          Feature files

          + status
          + trades
             record_a_new_trade.feature 
      + templates                         Freemarker templates and properties files                
Adding the Cucumber 4 dependency

Serenity seamlessly supports both Cucumber 2.x and Cucumber 4. However, this flexibility requires a little tweaking in the build dependencies.

If you are using Maven, you need to do the following:

  • exclude the default cucumber-core dependency from your serenity-core dependency
  • Replace your serenity-cucumber dependency with the serenity-cucumber4 dependency
  • Add dependencies on the Cucumber 4.x version of cucumber-java and cucumber-junit into your project

An example of the correctly configured dependencies is shown below:

<dependency>
    <groupId>net.serenity-bdd</groupId>
    <artifactId>serenity-core</artifactId>
    <version>2.0.40</version>
    <scope>test</scope>
    <exclusions>
        <exclusion>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-core</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>net.serenity-bdd</groupId>
    <artifactId>serenity-cucumber4</artifactId>
    <version>1.0.5</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-java</artifactId>
    <version>4.2.0</version>
</dependency>
<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-junit</artifactId>
    <version>4.2.0</version>
</dependency>

If you are using Gradle, you need to ensure that the 4.x version of cucumber-core is used using the resolutionStrategy element, and also add the Cucumber 4.x version of cucumber-java and cucumber-junit dependencies as mentioned above:

configurations.all {
    resolutionStrategy {
        force "io.cucumber:cucumber-core:4.2.0"
    }
}

dependencies {
    testCompile "net.serenity-bdd:serenity-core:2.0.40",
                "net.serenity-bdd:serenity-cucumber4:1.0.4",
                "io.cucumber:cucumber-core:4.2.0",
                "io.cucumber:cucumber-junit:4.2.0"
}

In the rest of this article, we will walk through some of the highlights of both versions. Let’s start off with the version on the master branch, which uses lightweight page objects and actions.

A simple GET scenario

The project comes with two simple scenarios, one that illustrates a GET, and a second that illustrates a POST.

The first scenario exercises the /api/status endpoint:

  Scenario: Application status end-point
    Given the application is running
    When I check the application status
    Then the API should return "Serenity REST Starter project up and running"

The glue code for this scenario illustrates the layered approach we find works well for both web and non-web acceptance tests. The glue code is responsible for orchestrating calls to a layer of more business-focused classes, which perform the actual REST calls.

    @Steps
    ApplicationStatus theApplication;

    @Given("the application is running")
    public void the_application_is_running() {
        assertThat(theApplication.currentStatus()).isEqualTo(RUNNING);
    }

    @When("I check the application status")
    public void i_check_the_application_status() {
        theApplication.readStatusMessage();
    }

The actual REST calls are performed using RestAssured in the action classes, like ApplicationStatus here. These use either RestAssured (if we don’t want the queries to appear in the reports) or SerenityRest (if we do):

public class ApplicationStatus {

    public AppStatus currentStatus() {
        int statusCode = RestAssured.get(STATUS.getUrl()).statusCode();
        return (statusCode == 200) ? AppStatus.RUNNING : AppStatus.DOWN;
    }

    @Step("Get current status message")
    public void readStatusMessage() {
        SerenityRest.get(STATUS.getUrl());
    }
}

In steps that perform assertions, we can also use the SerenityRest.restAssuredThat() helper method, which lets us make a RestAssured assertion on the last response the server sent us:

    @Then("the API should return {string}")
    public void the_API_should_return(String expectedMessage) {
        restAssuredThat(lastResponse -> lastResponse.body(equalTo(expectedMessage)));
    }
A more complex scenario

The other sample scenario performs a POST query:

Feature: Record a new trade

  Scenario: Each trade has a unique ID
    Given the following trade:
      | security | buySell | quantity | priceInCents |
      | APPL     | BUY     | 10       | 10000        |
    When I record the trade
    Then the recorded trade should include the following details:
      | security | buySell | quantity | priceInCents | totalCostInCents |
      | APPL     | BUY     | 10       | 10000        | 100000           |

The Given step uses a Freemarker template to merge the data in the Cucumber table with values defined in a properties file – to see how this works in detail, have a look at the MergeFrom class.

    @Given("the following trade:")
    public void the_folowing_trade(List<Map<String, String>> tradeDetails) throws IOException {

        trade = MergeFrom.template("templates/trade.json")
                         .withDefaultValuesFrom(FieldValues.in("templates/standard-trade.properties"))
                         .withFieldsFrom(tradeDetails.get(0));
    }

Once the message to be posted has been prepared, we use another action class (tradingSystem) to perform the post:

    @Steps
    RecordNewTrade recordNewTrade;
   
    @When("I record the trade")
    public void i_record_the_trade() {
        recordNewTrade.withDetails(trade);
    }

The RecordNewTrade class is responsible for posting this query to the end point, as shown below:

public class RecordNewTrade {

    @Step("Record a new trade")
    public void withDetails(String trade) {
        SerenityRest.given()
                .contentType("application/json")
                .header("Content-Type", "application/json")
                .body(trade)
                .when()
                .post(WebServiceEndPoints.TRADE.getUrl());
    }
}

The last step checks that the total cost has been recorded correctly in the trade. In a real project, this would typically be implemented via another REST call or by a database query, but here we are illustrating how we can get and compare tabular data from JSON responses.

    @Steps
    TradeResponse theTradeDetails;
  
    @Then("the recorded trade should include the following details:")
    public void the_recorded_trade_should_contain_the_following_details(List<Map<String, String>> tradeDetails) {
        restAssuredThat(response -> response.statusCode(200));

        Map<String, String> expectedResponse = tradeDetails.get(0);
        Map<String, String> actualResponse = theTradeDetails.returned();

        assertThat(actualResponse).containsAllEntriesOf(expectedResponse);
    }

The TradeResponse class is responsible for retrieving the latest REST response and converting it to a map of strings.

public class TradeResponse {
    public Map<String, String> returned() {
       return mapOfStringsFrom(SerenityRest.lastResponse().getBody().as(Map.class));
    }

    private Map<String,String> mapOfStringsFrom(Map<String, Object> map) {
        return map.entrySet()
                .stream()
                .collect(toMap(Map.Entry::getKey,
                        entry -> entry.getValue().toString()));
    }
}
Living documentation

You can generate full Serenity reports by running mvn clean verify. This includes both the living documentation from the feature files:

And also details of the REST requests and responses that were executed during the test:

Want to learn more?

For more information about Serenity BDD, you can read the Serenity BDD Book, the official online Serenity documentation source. Other sources include:

The post Getting started with REST API testing with Serenity and Cucumber appeared first on John Ferguson Smart.

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Serenity BDD is a library that makes it easier to write high quality automated acceptance tests, with powerful reporting and living documentation features. It has strong support for both web testing with Selenium, and API testing using RestAssured.

Serenity strongly encourages good test automation design, and supports several design patterns, including classic Page Objects, the newer Lean Page Objects/ Action Classes approach, and the more sophisticated and flexible Screenplay pattern.

The latest version of Serenity supports both Cucumber 2.4 and the more recent Cucumber 4.x. Cucumber 4 is not backward compatible with Cucumber 2. This article walks you through how to get started with Serenity and Cucumber 4, and also gives you a quick introduction to some of Cucumber 4’s new features.

The starter project

The best place to start with Serenity and Cucumber is to clone or download the starter project on Github (https://github.com/serenity-bdd/serenity-cucumber4-starter). This project gives you a basic project setup, along with some sample tests and supporting classes. There are two versions to choose from. The master branch uses a more classic approach, using action classes and lightweight page objects, whereas the screenplay branch shows the same sample test implemented using Screenplay.

The project directory structure

The project has build scripts for both Maven and Gradle, and follows the standard directory structure used in most Serenity projects:

src
  + main
  + test
    + java                        Test runners and supporting code
    + resources
      + features                  Feature files
    + search                      Feature file subdirectories
        search_by_keyword.feature

  + webdriver                     Bundled webdriver binaries
    + linux
    + mac
    + windows
       chromedriver.exe           OS-specific Webdriver binaries
       geckodriver.exe
Adding the Cucumber 4 dependency

Serenity seamlessly supports both Cucumber 2.x and Cucumber 4. However, this flexibility requires a little tweaking in the build dependencies.

If you are using Maven, you need to do the following:

  • exclude the default cucumber-core dependency from your serenity-core dependency
  • Replace your serenity-cucumber dependency with the serenity-cucumber4 dependency
  • Add dependencies on the Cucumber 4.x version of cucumber-java and cucumber-junit into your project

An example of the correctly configured dependencies is shown below:

<dependency>
    <groupId>net.serenity-bdd</groupId>
    <artifactId>serenity-core</artifactId>
    <version>2.0.39</version>
    <scope>test</scope>
    <exclusions>
        <exclusion>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-core</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>net.serenity-bdd</groupId>
    <artifactId>serenity-cucumber4</artifactId>
    <version>1.0.5</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-java</artifactId>
    <version>4.2.0</version>
</dependency>
<dependency>
    <groupId>io.cucumber</groupId>
    <artifactId>cucumber-junit</artifactId>
    <version>4.2.0</version>
</dependency>

If you are using Gradle, you need to ensure that the 4.x version of cucumber-core is used using the resolutionStrategy element, and also add the Cucumber 4.x version of cucumber-java and cucumber-junit dependencies as mentioned above:

configurations.all {
    resolutionStrategy {
        force "io.cucumber:cucumber-core:4.2.0"
    }
}

dependencies {
    testCompile "net.serenity-bdd:serenity-core:2.0.39",
                "net.serenity-bdd:serenity-cucumber4:1.0.5",
                "io.cucumber:cucumber-core:4.2.0",
                "io.cucumber:cucumber-junit:4.2.0"
}

In the rest of this article, we will walk through some of the highlights of both versions. Let’s start off with the version on the master branch, which uses lightweight page objects and actions.

The Cucumber 4 sample scenario

Both variations of the sample project uses the sample Cucumber scenario. In this scenario, Sergey (who likes to search for stuff) is performing a search on the DuckDuckGo search engine:

Feature: Search by keyword

  Scenario: Searching for a term
    Given Sergey is on the DuckDuckGo home page
    When he searches for "cucumber"
    Then all the result titles should contain the word "cucumber"

This scenario lets us explore a few of the new Cucumber 4 expressions. Cucumber 4 supports both the classic regular expressions, and the new Cucumber Expressions, which are more readable albeit not as powerful in some cases.

The glue code for this scenario uses both regular expressions and cucumber expressions. The glue code looks this this:

    @Given("^(?:.*) is on the DuckDuckGo home page")  
    public void i_am_on_the_DuckDuckGo_home_page() {
        navigateTo.theDuckDuckGoHomePage();
    }

    @When("(s)he searches for {string}")               
    public void i_search_for(String term) {
        searchFor.term(term);
    }

    @Then("all the result titles should contain the word {string}")
    public void all_the_result_titles_should_contain_the_word(String term) {
        assertThat(searchResult.titles())
                .matches(results -> results.size() > 0)
                .allMatch(title ->  
                         textOf(title).containsIgnoringCase(term));
    }

The @Given step uses a regular expression; the action class approach we use here is action-centric, not actor-centric, so we ignore the name of the actor.

The @When and @Then steps uses Cucumber expressions, and highlights two useful features. Rather than using a regular expression to match the search term, we use the more readable Cucumber expression {string}. This matches a single or double-quoted string (the quotes themselves are dropped). Cucumber 4 also supports other typed expressions, such as {int}, {word}, and {float}.

Parentheses can be used to indicate optional text, so “(s)he” will match both “he” and “she”. We could also write this using a slash: “she/he”.

Lean Page Objects and Action Classes

The glue code shown above uses Serenity step libraries as action classes to make the tests easier to read, and to improve maintainability.

These classes are declared using the Serenity @Steps annotation, shown below:

    @Steps
    NavigateTo navigateTo;

    @Steps
    SearchFor searchFor;

    @Steps
    SearchResult searchResult;

The @Stepsannotation tells Serenity to create a new instance of the class, and inject any other steps or page objects that this instance might need.

Each action class models a particular facet of user behaviour: navigating to a particular page, performing a search, or retrieving the results of a search. These classes are designed to be small and self-contained, which makes them more stable and easier to maintain.

The NavigateTo class is an example of a very simple action class. In a larger application, it might have some other methods related to high level navigation, but in our sample project, it just needs to open the DuckDuckGo home page:

public class NavigateTo {

    DuckDuckGoHomePage duckDuckGoHomePage;

    @Step("Open the DuckDuckGo home page")
    public void theDuckDuckGoHomePage() {
        duckDuckGoHomePage.open();
    }
}

It does this using a standard Serenity Page Object. Page Objects are often very minimal, storing just the URL of the page itself:

@DefaultUrl("https://duckduckgo.com")
class DuckDuckGoHomePage extends PageObject {}

The second class, SearchFor, is an interaction class. It needs to interact with the web page, and to enable this, we make the class extend the Serenity UIInteractionSteps. This gives the class full access to the powerful Serenity WebDriver API, including the $() method used below, which locates a web element using a By locator or an XPath or CSS expression:

public class SearchFor extends UIInteractionSteps {

    @Step("Search for term {0}")
    public void term(String term) {
        $(SearchForm.SEARCH_FIELD).clear();
        $(SearchForm.SEARCH_FIELD).type(term);
        $(SearchForm.SEARCH_BUTTON).click();
    }
} 

The SearchForm class is typical of a light-weight Page Object: it is responsible uniquely for locating elements on the page, and it does this by defining locators or occasionally by resolving web elements dynamically.

class SearchForm {
    static By SEARCH_FIELD = By.cssSelector(".js-search-input");
    static By SEARCH_BUTTON = By.cssSelector(".js-search-button");
}

The last step library class used in the step definition code is the SearchResult class. The job of this class is to query the web page, and retrieve a list of search results that we can use in the AssertJ assertion at the end of the test. This class also extends UIInteractionSteps and

public class SearchResult extends UIInteractionSteps {
    public List<String> titles() {
        return findAll(SearchResultList.RESULT_TITLES)
                .stream()
                .map(WebElementFacade::getTextContent)
                .collect(Collectors.toList());
    }
}

The SearchResultList class is a lean Page Object that locates the search result titles on the results page:

class SearchResultList {
    static By RESULT_TITLES = By.cssSelector(".result__title");
}

The main advantage of the approach used in this example is not in the lines of code written, although Serenity does reduce a lot of the boilerplate code that you would normally need to write in a web test. The real advantage is in the use of many small, stable classes, each of which focuses on a single job. This application of the Single Responsibility Principle goes a long way to making the test code more stable, easier to understand, and easier to maintain.

The Screenplay starter project

If you prefer to use the Screenplay pattern, or want to try it out, check out the screenplay branch instead of the master branch. In this version of the starter project, the same scenario is implemented using the Screenplay pattern.

The Screenplay pattern describes tests in terms of actors and the tasks they perform. Tasks are represented as objects performed by an actor, rather than methods. This makes them more flexible and composable, at the cost of being a bit more wordy. Here is an example:

    @Before
    public void setTheStage() {
        OnStage.setTheStage(new OnlineCast());
    }

    @Given("^(.*) is on the DuckDuckGo home page")
    public void on_the_DuckDuckGo_home_page(String actor) {
        theActorCalled(actor).attemptsTo(
        NavigateTo.theDuckDuckGoHomePage()
    );
    }

    @When("she/he searches for {string}")
    public void search_for(String term) {
        theActorInTheSpotlight().attemptsTo(
             SearchFor.term(term) 
    );
    }

    @Then("all the result titles should contain the word {string}")
    public void all_the_result_titles_should_contain_the_word(String term) {
        theActorInTheSpotlight().should(
                seeThat("search result titles",
                        SearchResult.titles(),
                    hasSize(greaterThan(0))),
                seeThat("search result titles",
                        SearchResult.titles(),
                    everyItem(containsIgnoringCase(term)))
        );
    }

In both approaches, the Page Objects very close or identical. The differences are mainly in the action classes. Screenplay classes emphasise reusable components and a very readable declarative style, whereas Lean Page Objects and Action Classes opt for a more imperative style.

The NavigateTo class performs the same role as it’s equivalent in the Lean Page Object/Action Class version, and looks quite similar:

public class NavigateTo  {

    public static Performable theDuckDuckGoHomePage() {
        return Task.where("{0} opens the DuckDuckGo home page",
                Open.browserOn().the(DuckDuckGoHomePage.class)
        );
    }
} 

The SearchFor class is also similar: it is shown below:

public class SearchFor {

    public static Performable term(String term) {
        return Task.where("{0} attempts to search for #term",
                Clear.field(SearchForm.SEARCH_FIELD),
            Enter.theValue(term).into(SearchForm.SEARCH_FIELD),
                Click.on(SearchForm.SEARCH_BUTTON)
        ).with("term").of(term);
    }
}

In Screenplay, there is a clear distinction between actions (which change the system state) and questions (which read the system state). In Screenplay, we fetch the search results using a Question class, like this:

public class SearchResult {
    public static Question<List<String>> titles() {
        return actor ->  
                 TextContent.of(SearchResultList.RESULT_TITLES)
                            .viewedBy(actor)
                            .asList();
    }
}

The Screenplay DSL is rich and flexible, and well suited to teams working on large test automation projects with many team members, and who are reasonably comfortable with Java and design patterns. The Lean Page Objects/Action Classes approach proposes a gentler learning curve, but still provides significant advantages in terms of maintainability and reusability.

Executing the tests

To run the sample project, you can either just run the CucumberTestSuite test runner class, or run either mvn verify or gradle test from the command line.

By default, the tests will run using Chrome. You can run them in Firefox by overriding the driver system property, e.g.

$ mvn clean verify -Ddriver=firefox

Or

$ gradle clean test -Pdriver=firefox

The test results will be recorded in the target/site/serenity directory.

Simplified WebDriver configuration and other Serenity extras

The sample projects both use some Serenity features which make configuring the tests easier. In particular, Serenity uses the serenity.conf file in the src/test/resources directory to configure test execution options.

Webdriver configuration

The WebDriver configuration is managed entirely from this file, as illustrated below:

webdriver {
    driver = chrome
}
headless.mode = true

chrome.switches="""--start-maximized;--test-type;--no-sandbox;--ignore-certificate-errors;
                   --disable-popup-blocking;--disable-default-apps;--disable-extensions-file-access-check;
                   --incognito;--disable-infobars,--disable-gpu"""

The project also bundles some of the WebDriver binaries that you need to run Selenium tests in the src/test/resources/webdriver directories. These binaries are configured in the drivers section of the serenity.conf config file:

drivers {
  windows {
    webdriver.chrome.driver = "src/test/resources/webdriver/windows/chromedriver.exe"
    webdriver.gecko.driver = "src/test/resources/webdriver/windows/geckodriver.exe"
  }
  mac {
    webdriver.chrome.driver = "src/test/resources/webdriver/mac/chromedriver"
    webdriver.gecko.driver = "src/test/resources/webdriver/mac/geckodriver"
  }
  linux {
    webdriver.chrome.driver = "src/test/resources/webdriver/linux/chromedriver"
    webdriver.gecko.driver = "src/test/resources/webdriver/linux/geckodriver"
  }
}

This configuration means that development machines and build servers do not need to have a particular version of the WebDriver drivers installed for the tests to run correctly.

Environment-specific configurations

We can also configure environment-specific properties and options, so that the tests can be run in different environments. Here, we configure three environments, dev, staging and prod, with different starting URLs for each:

environments {
  default {
    webdriver.base.url = "https://duckduckgo.com"
  }
  dev {
    webdriver.base.url = "https://duckduckgo.com/dev"
  }
  staging {
    webdriver.base.url = "https://duckduckgo.com/staging"
  }
  prod {
    webdriver.base.url = "https://duckduckgo.com/prod"
  }
}

You use the environment system property to determine which environment to run against. For example to run the tests in the staging environment, you could run:

$ mvn clean verify -Denvironment=staging

See this article for more details about this feature.

Want to learn more?

For more information about Serenity BDD, you can read the Serenity BDD Book, the official online Serenity documentation source. Other sources include:

The post Getting started with Serenity BDD and Cucumber 4 appeared first on John Ferguson Smart.

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Page Objects are one of the most widely used test automation design patterns around. Most QA engineers have used a variation of Page Objects at some point. However, it is often misunderstood and used poorly, which can result in test automation code that is fragile and hard to maintain.

In this article, we will look at some of the limits and pitfalls of the Page Objects pattern, as well as some approaches that can help you ensure that your Page Objects don’t lead you to a maintenance nightmare.

Origins of the Page Objects pattern

The basic idea of Page Objects is sound – hide the logic about how you find the elements of a page (for example, id fields or XPath or CSS selectors) from how you interact with these elements (what value your enter into a field for example). The idea is to keep the selector logic in one place, so that if a page element changes, you only need to update the code in one place.

The Page Objects pattern has been around for a while. In the context of Selenium WebDriver testing, the model was originally proposed by Simon Steward as a way to introduce testers to a more maintainable, Object Oriented approach to test automation.

Pages Objects are indeed much more maintainable than unstructured Selenium test scripts, and work well for simple scenarios. But for more complex UIs and more complex scenarios, they need to be used together with other design patterns to ensure that the Page Objects themselves do not become a maintenance overhead.

Limits of the Page Object model

Many articles have been written on the limits of the Page Object model. Here are a few of the key limitations:

  • They don’t work well for modern UI designs: web applications today are much more complex and more interactive than they were a decade ago. Rich UI components are used across multiple pages, and on any given page users can perform a wide variety of actions. When you try to model each page as a single Page Object class, you end up with large and complex classes that can be hard to understand and harder to maintain.
  • They make you think at the wrong level of abstraction: A more subtle problem is that using Page Objects gives you a skewed perception of your test automation architecture. When you build your test framework around Page Objects, you tend to see everything in terms of interactions with a web page. And this leads to tests that do everything via the UI, even tasks which could be done more reliably and more quickly by calling a REST API or querying a database.
  • They can easily become bloated and hard to maintain: When all you have is Page Objects, all your logic goes in your Page Objects. Page Objects often have a mix of code that locates elements on a page (for example, a quantity dropdown, a size radio button, and an Add To Cart button) and that interacts with these elements (e.g a addDressesToCart() method). There ends up being a lot of code doing a lot of different things in a single class. This makes the class harder to understand and to maintain. It also means that when you modify one of the Page Object methods for your own test, you may well break it for other tests. Large Page Object classes undergo a lot of churn, and this inevitably introduces risk.
Tips for writing good Page Objects

Despite these limitations, if your current test automation suite relies on Page Objects, do not despair! There are a few simple things you can do to help ensure that your Page Objects do not become a liability.

Think Page Components, not Page Objects

If your Page Objects are a bit on the heavy side, one of the simplest improvements is to break them up into smaller classes. If you have a complex page, don’t try to place every field into a single class. Instead, you can use the concept of Page Components. A Page Component represents a specific part of the page that helps the user perform a specific task. A login form, a navigation hierarchy, a search result list, or the details about the current user: all of these would make great Page Components.

For example, consider the following web page:

On this page, we can identify at least three different sections, each of which could be represented by independent components:

  • The menu bar at the top of the page
  • The “Plan a journey” search panel
  • The “My journeys” history box
  • The Search box in the top right hand corner

Some of these, such as the menu bar and the search box, appear on many pages. And other components, such as the travel preferences shown below, only appear in certain situations. But most importantly, the test code that interacts with each of these components can be developed and maintained independently.

In coding terms, a page component looks just like a page object, only much smaller. Below you can see an example of a Page Component class for the menu bar. (The examples in the rest of this article will use Serenity BDD Page Objects, but the principles apply equally to any Page Object implementation).

class MenuBar extends PageObject {
    
    @FindBy(css = ".collapsible-menu")
    private WebElement menuBar;

    public void selectMenuItem(String menuItem) {
        menuBar.findElement(By.linkText(menuItem)).click();
    }
}
Keep business logic out of your Page Objects

Another reason Page Objects become large and hard to maintain is that they try to do too many things.

For example, suppose we wanted to check that a particular menu item was enabled for certain users, and disabled for others. We could add some more methods to the MenuBar class to perform these checks:

class MenuBar extends PageObject {

    @FindBy(css = ".collapsible-menu")
    private WebElement menuBar;

    public void selectMenuItem(String menuItem) {
        menuBar.findElement(By.linkText(menuItem))
           .click();
    }
    
    public void checkMenuItemIsEnabled(String menuItem) {
        assertTrue(
            menuBar.findElement(By.linkText(menuItem))
                   .isEnabled()
        );
    }

    public void checkMenuItemIsDisabled(String menuItem) {
        assertFalse(
            menuBar.findElement(By.linkText(menuItem))
                   .isEnabled()
        );
    }
}

A test using this class would check the state of the menu item by calling this page object method:

    @Test
    public void plan_my_journey_is_available_for_all_users() {
        navigate.toTheTFLHomePage();
        menuBar.checkMenuItemIsEnabled("Plan a journey");
    }

However, adding logic into a Page Object class is another road to complexity and maintenance headaches. You would also need to add methods for other checks: what if we need to check that an element is hidden entirely?

A cleaner approach is to make Page Object classes responsible for doing a single job: locating elements on the page. The MenuBar Page Object could be limited to locating menu items by name:

public class MenuBar extends PageObject {

    @FindBy(css = ".collapsible-menu")
    private WebElement menuBar;

    public WebElement itemFor(String menuItem) {
        return menuBar.find(By.linkText(menuItem));
    }
}

The test code could now refer to this element directly, to make whatever assertions it sees fit. In the following example, we use AssertJ to check that a given menu item is both displayed and enabled:

    @Test
    public void plan_my_journey_is_available_for_all_users() {
        navigate.toTheTFLHomePage();
        assertThat(menuBar.itemFor("Plan a journey"))
                          .matches(WebElement::isDisplayed)
                          .matches(WebElement::isEnabled);
    }
Model user behaviour not user interfaces

In many test automation projects built around Page Objects, the Page Objects are manipulated directly in the tests. For example, the following code uses Page Objects in a fairly typical way:

    TFLHomePage tflHomePage;
    TFLSearchResultPage tflSearchResultPage;
    TFLStatusUpdatesPage tflStatusUpdatesPage;

    @Test
    public void trips_between_two_stations() {
        tflHomePage.open();
        tflHomePage.selectFrom("Paddington Underground Station");
        tflHomePage.selectTo("Liverpool Street");
        tflHomePage.clickOnPlanMyJourney();
        tflSearchResultPage.waitForResults();

        assertThat(tflSearchResultPage.getJourneyResultFrom())
            .isEqualTo("Paddington Underground Station");
        assertThat(tflSearchResultPage.getLastStop())
            .endsWith("to Liverpool Street");
    }

This code wraps hides the location logic for the page elements inside page objects, as prescribed by the Page Object pattern. It cleanly separates WebDriver calls from the test code itself. And yet tests like this still quickly become hard to maintain.

It is not hard to see why. Imagine we need to write another, similar test, one where the user prefers to travel by bus.

    @Test
    public void choose_a_preferred_transport_mode() {
        tflHomePage.open();
        tflHomePage.clickOnEditPreferences();
        tflHomePage.deselectAllTravelModes();
        tflHomePage.selectTravelMode("Bus");
        tflHomePage.selectFrom("Paddington Underground Station");
        tflHomePage.selectTo("Liverpool Street");
        tflHomePage.clickOnPlanMyJourney();
        tflSearchResultPage.waitForResults();

        assertThat(tflSearchResultPage.getItinerary())
            .contains("205 bus to Liverpool Street Station");
    }

The code in this test is almost, but not quite, identical to the previous one. In a real-world test suite, there would be many others like it. Each describes how the user interacts with the user interface in great detail. But this makes the tests verbose and noisy, which in turn makes them hard to read and harder to maintain.

They are also fragile. The Page Object model keeps the locators for each field in a central place, which is good. But our tests are tightly coupled to the user interface. If the broader UI design changes, the impact is spread across many tests.

For instance, imagine a change where, instead of simply entering a departure and destination station, the user needs to type the name of the station, and then select the station in a dynamic dropdown:

The code to automate this interaction might now look like this:

    tflHomePage.selectFrom("Paddington Underground Station");
    tflHomePage.clickOnStopInDropdownNamed(
                           "Paddington Underground Station");
    tflHomePage.selectTo("Liverpool Street");
    tflHomePage.clickOnStopInDropdownNamed("Liverpool Street");

This change is not hard in itself, but using a classic Page Object model, it would impact every single test that exercises the “Plan a journey” feature. And the more code you need to change, the more likely you are to make a mistake and break a test.

Encapsulate interaction logic inside step methods

A more maintainable and more robust approach is not just to model how the user interacts with the application, but to model what the user is trying to do in business terms.

Let’s see what we mean by this. In the first test we saw, the user is performing three tasks:

  • Navigate the TFL home page
  • Plan a journey from Paddington to Liverpool Street Station
  • View the proposed itinerary.

We could make the original test more readable by reorganising the interactions into “step” methods to reflect this breakdown:

    @Test
    public void choose_a_preferred_transport_mode() {

        navigateToTFLHomePage;

        planAJourneyBetween("Paddington Underground Station",
                            "Liverpool Street");

        assertThat(tflSearchResultPage.getJourneyResultFrom())
            .isEqualTo("Paddington Underground Station");
        assertThat(tflSearchResultPage.getLastStop())
            .endsWith("to Liverpool Street");
    }

This way, the UI interactions for each step are grouped in one place, making them easier to maintain and making the test code easier to understand. For example, the planAJourneyBetween() method would look like this:

    public void planAJourneyBetween(String departure, 
                                    String destination) {
        tflHomePage.selectFrom(departure);
        tflHomePage.clickOnStopInDropdownNamed(departure);
        tflHomePage.selectTo(destination);
        tflHomePage.clickOnStopInDropdownNamed(destination);
        tflSearchResultPage.waitForResults();
    }
Keep step methods in “action” classes

Breaking a test into smaller step methods is a useful technique, but you can only use these methods within a single test class. Oftentimes, these step methods can be used across many tests, or even across multiple projects.

An approach that scales better is to put the step methods in their own distinct classes, rather than including them in the test classes themselves.This makes it easier to reuse the step methods in different tests. We call these classes Action classes.

Serenity BDD provides special support for this pattern, but you can use the same approach with any framework. An example of a class that encapsulates navigation tasks can be seen below:

public class Navigate {
    TFLHomePage homePage;
    MenuBar menuBar;

    public void toTheTFLHomePage() {
        homePage.open();
    }

    public void toMenuItem(String menuItem) {
        menuBar.itemFor(menuItem).click();
    }
}

Now, rather than adding a method to the test case, we would use an instance of theNavigate class. In Serenity BDD, we can use the @Steps annotation to instantiate the navigate field for us:

   @Steps
   Navigate navigate;

   @Test
   public void choose_a_preferred_transport_mode() {
       navigate.ToTFLHomePage;
       ...
Use action classes to model user tasks

There are many ways to organise step methods into classes. One approach is to organise steps by user role, so that each step role represents a different user. However, different users may want to perform the same tasks.

A more flexible approach is to group step methods by business task. For example, we could have a PlanMyJourneySteps class, which includes methods related to different ways a user can plan a journey between two stations.

Using Serenity BDD, the PlanMyJourneySteps might look something like this:

public class PlanMyJourneySteps extends UIInteractionSteps {

    @Step("Plan my journey from {0} to {1}")
    public void between(String from, String to) {
        selectStation(FROM_STATION, from);
        selectStation(TO_STATION, to);
        clickOnPlanYourJourney();
    }

    @Step("Plan my journey from {0} to {1} on {2}")
    public void between(String from,
                        String to,
                        String departureDate) {
        selectStation(FROM_STATION, from);
        selectStation(TO_STATION, to);
        changeDepartureDateTo(departureDate);
        clickOnPlanYourJourney();
    }
    ...

Let’s look at the implementation of these methods more closely.

Lean Page Objects and Action classes

We saw earlier how it makes sense to keep our Page Objects clear of business logic, and have them focus on locating elements on the web page. It is the step methods that orchestrate the actual interactions with the page. I like to call these Lean Page Objects.

There are two ways we can write Lean Page Objects. One approach is to use @FindBy-annotated WebElement fields and to write wrapper methods which interact with these fields, like this:

public class ChooseStationsPageComponent extends PageObject {

    @FindBy(id="InputFrom")
    WebElement fromStation;

    @FindBy(id="InputTo")
    WebElement toStation;

    public static String STATION_SUGGESTION = 
      "//span[contains(@class,'tt-suggestion')][contains(.,'%s')]";

    public void selectFrom(String from) {
        fromStation.sendKeys(from);
    }

    public void selectTo(String to) {
        toStation.sendKeys(to);
    }

    public void clickOnStopInDropdownNamed(String station) {
        String locator =  String.format(STATION_SUGGESTION, 
                                        station);
        findBy(locator).click();
    }
}

The action class would declare the page object and call these methods:

public class ChooseStationSteps {

    ChooseStationsPageComponent chooseStations;

    @Step
    private void selectDepartureStation(String stationName) {
        chooseStations.selectFrom(stationName);
        chooseStations.clickOnStopInDropdownNamed(stationName);
    }

    @Step
    private void selectDestinationStation(String stationName) {
        chooseStations.selectTo(stationName
        chooseStations .clickOnStopInDropdownNamed(stationName);
    }
}

The second approach is to have Page Objects that are responsible solely for locating web elements, and have the methods in the action classes manipulate these elements. So rather than exposing methods which manipulate web elements, the Page Objects expose only locators.

Using this approach, our Page Object could look like this:

public class PlanMyJourneyUI {
    public static By FROM_STATION = By.id("InputFrom");
    public static By TO_STATION = By.id("InputTo");
    
    private static String STATION_SUGGESTION = 
      "//span[contains(@class,'tt-suggestion')][contains(.,'%s')]";

    public static By stationSuggestionFor(String stationName) {
        return By.xpath(
           String.format(STATION_SUGGESTION, stationName)
        );
    }
}

And the UI Interaction step class might look like this:

public class ChooseStationSteps extends UIInteractionSteps {

    @Step
    private void selectDepartureStation(String stationName) {
        find(FROM_STATION).sendKeys(stationName);
        find(stationSuggestionFor(stationName)).click();
    }

    @Step
    private void selectDestinationStation(String stationName) {
        find(TO_STATION).sendKeys(stationName);
        find(stationSuggestionFor(stationName)).click();
    }
}

In Serenity BDD, the UIInteractionSteps class marks a class as a WebDriver-enabled action class, and gives the methods full access to the Serenity WebDriver API. The @Step annotation tells Serenity BDD to include this step (with corresponding screenshots) in the test report.

The two approaches are similar, but the second approach tends to result in leaner, more flexible code and a better separation of concerns.

Separate actions and questions

So far our step methods have focused on doing things to the UI – clicking buttons, selecting values in dropdown lists, and so on. The other type of step method involves reading values from the UI (or from somewhere else) to check whether the application has done what we expect. You might call these methods “query” methods.

Some folks prefer to keep query methods in the same classes as the action methods. I like to put them in a separate class, as it makes the test code a little more readable. It also gives you a bit..

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Serenity is primarily designed to report the results of automated acceptance tests. However, there are times when some tests need to be done manually. And it is useful to report these tests in the overall test reports, to get a broader picture of test coverage.

To make this easier, Serenity with Cucumber provides some support for recording and reporting manual test results.

You can mark a test as a manual test using the @manual tag, e.g.

@Manual 
Scenario: Monitoring a new low risk customer 
    Given Joe is a new customer 
    And Joe is considered a low risk customer 
    When he is onboarded 
    Then his account should be scheduled for review in 3 months time

This will appear in the reports as a manual test, as shown below.

Figure 1. A manual test reported in Serenity

By default, manual tests are reported as “pending”, like the one above. The individual steps will be marked as ignored, as they are just there for documentation purposes.

You can override this status, and mark a test explicitly as a passing or failing test, like this:

@Manual:Passed
Scenario: Monitoring a new low risk customer 
    Given Joe is a new customer 
    And Joe is considered a low risk customer 
    When he is onboarded 
    Then his account should be scheduled for review in 3 months time

Or if you want to report that a manual test was unsuccessful:

@Manual:Failed
Scenario: Monitoring a new low risk customer
Given Joe is a new customer
And Joe is considered a low risk customer
When he is onboarded
Then his account should be scheduled for review in 3 months time

The test will then appear as both a manual and a failing test:

Figure 2. A failing manual test reported in Serenity

If you need to provide more details about the test failure, you can add a note starting with the “Failure:” keyword underneath the scenario title, e.g.

@Manual
Scenario: Monitoring a new low risk customer
Failure:Joe is showing as a high-risk customer

Given Joe is a new customer
And Joe is considered a low risk customer
When he is onboarded
Then his account should be scheduled for review in 3 months time

This error message will then appear in the report:

Figure 3. A failing manual test including an error message.

Manual test results also appear in the overall test reports, where they are represented in a lighter shade of the normal test result colour:

Figure 4. Manual tests appearing in a summary report

Note that you should use this feature with caution, as marking a manual test as passing may be misleading: if you are running your Serenity tests on a CI server, you cannot safely say that they were manually tested with the version that was built on the build server. For this reason, manual test results should be considered as indicative, not definitive.

Although Manual tests can have steps (like the one above), they are not really supposed to have step definitions. If Cucumber finds a step definition for a step, it will execute it, and this may not be what you intend if you mark a test as manual.

The post Reporting Manual Test Results in Serenity BDD appeared first on John Ferguson Smart.

Read for later

Articles marked as Favorite are saved for later viewing.
close
  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Separate tags by commas
To access this feature, please upgrade your account.
Start your free month
Free Preview