How to Write Good Gherkin

How to Write Good Gherkin

A Guide For the Essential Element of BDD

By David Robertus

I have been using BDD, Gherkin and Cucumber for only about two years.  In that short span, though, I've acted as a Product Owner, a developer and as an Engineer in Test/QA, so I like to think I have had a fairly broad exposure to the various perspectives that Gherkin is intended to pull together.  I have run into some common patterns for using Cucumber and Gherkin, and along with its use I have made and observed a wide range of mistakes.  This is my effort to provide a “style guide” to an industry that is largely not well versed in having a writing technique or using grammatical language as a guide to software design.

Also, I am aware of the irony of using poor grammar in the title.  A better one would be “How to Write Gherkin Well”, or “Well Written Gherkin in BDD”, something to that effect, but that is a hair even I find a little useless to split.
 

When I was in high school we had a visitor to the physics class to give us advice on our college class selection.  One sentence still sticks in my mind: “Take all the English and Literature courses that you can; no matter how much you know about technology it will be wasted if you can’t communicate it”.  The older I get the truer and broader this statement becomes.  This is still a large, and common, gap in the education of many in the fields of Software Engineering, Product Management and in QA.  As Steve Jobs commented, “Technology alone is not enough. It’s technology married with the liberal arts, married with the humanities, that yields the results that makes our hearts sing…”  (excerpt at http://blogs.hbr.org/2011/08/steve-jobss-ultimate-lesson-fo/). Admittedly this statement seems somewhat melodramatic for the purposes of most businesses, but for the customer-centric business the evidence for it is quite apparent.

A Basic Example

When writing a feature or story for a product the first thing to keep in mind is the audience.  It is a shared document, to be a reference for Product, Development and QA. I have found that it is often the ideal input for a technical writer to produce customer facing documentation, and in some cases it is shared with the customer themselves.  It is not hard to envision the same document or incarnation thereof being used in technical support or training documentation.

Like a work of fiction, a story in Gherkin should bring an image to the reader’s mind.  The mental image that different readers of the same Gherkin story have in mind should be identical in functionality and preferably also data structure.  A well written story will do that- there will be no ambiguity as to the necessary functionality, and only cosmetic differences in ones mental implementation.  A poorly written story is one that does not accurately and unambiguously  reflect the underlying intentions of the feature.

Let’s start with a concrete example.  We need to write a feature that will define a web page used by customers.  For this example let us assume the page will select which Teddy Bear Conferences he or she plans to attend this year (let’s assume this is an important feature of the multi-billion dollar Teddy Bear Conference industry).

Lets start with the basics, the top level feature definition:

Feature: As a prospective attendee of a conference, I need to select the conferences I plan to attend and the role I plan to perform at each conference.

Hard to argue with that, right? Well, maybe, maybe not.  Let’s look at the variables we have here and the potential interplay between them.  Attendee, Conference, Role.  Will the page allow us to deal with multiple attendees, a group together?  How many Roles are there?  Can a single Attendee act in different Roles at different Conferences? There needs to be at least one Conference selected, but does the Teddy Bear Association impose a limit of some kind ( for example, “A member is not allowed to present a lecture at a conference more that twice a year due to issues of fairness and high demand for conference room space”)?  Each of these demands different UI arrangements, validation rules, and demands different database structures, all of which need to be discernible from this document.

These are the constraints that will be answered unambiguously and in a way that is portable across Product Owners, Development and QA.

Lets start with a basic one.

Scenario: User registers as Basic Attendee

        Given that the user is at the Conference Registration page

               And has selected Role of Attendee

               And the user has select one Conference city                

        When the user submits the page

        Then the user will be registered for that conference

              And the page will navigate to the Completed page

Here is where literary criticism is an underrated technical skill.  Let's dissect this scenario for clarity and ambiguity.

The first line looks good- we don’t necessarily need to know how we got to the Conference Registration page.  That is above and outside this feature.

The second line has some problems.  We have ambiguously defined the relationship of the Role to the Attendee.  Can they have many Roles? If so, one or many Roles per Conference? Is there a default Role?  From the step “And has selected Role of Attendee” implies that this requires active selection but it is not a default value.  This may not be desirable since the scenario is first and described as “Basic Attendee”, probably the most common usage. 

The third line has some ambiguity as well.  The phrase “selected one Conference city” doesn't demand a minimum or maximum.  It also indicates possible shortcomings of our definition of a Conference- a single Conference may need to be in a city, but the same city can host multiple times.  Do we need to add a date?

The order between the second and third line also suggests a limitation of the association between the customer, Role and Conference.  As it is written it implies that there is a one-to-one relation between customer and Role and that that Role applies to any Conferences that are selected.

Here is one of several valid configuration of database tables described by the above Scenario:

So, what do we REALLY mean? Let’s rewrite this scenario to better express the relationships we want to allow.

Scenario: User registers as Basic Attendee

        Given that the user is at the Conference Registration page

                And the user has selected at least one Conference city/date combination

                And has left the default “Attendee” Role for each selected Conference

        When the user submits the page

        Then the user will be registered for the conference(s)

                And the page will navigate to the Completed page

Now we have a much clearer mental picture of what our user interface and database must allow, validate and store. We can see that a Conference must consist of a city and a date. We see that there is a minimum number of conferences to select (one), but that there is no upper limit (yet).  We see that there is a default Role and that it is set to “Attendee”.  We see that the user can have a different Role at each Conference.
  

Here is the ONLY viable database configuration for the updated Scenario:

The lesson here is very simple- writing matters.  Phrasing matter.  Sentence order matters.  There is a reason that in Agile and BDD we talk about a “story”.  We are telling a story, and there is a difference between a well written story and a poorly written one.  While no BDD stories are likely to be considered for a Pulitzer, we already see the implications of a well written versus poorly written story.  While we have not implemented anything in code, we have a common, unambiguous mental picture of the relationships we want to describe.  A customer would know what he needs to do, a developer knows what the web page will need to support, what the business rules will need to validate, and how the database will need to be designed; QA will be able to give a thumbs up or down to the implementation.  Compare that with the ambiguity of the first version of our Scenario.  How many different versions of the same web page (and business layer and database) could be made that would “meet” the scenario.  Certainly more than one, and that is too many.  

There are a number of common mistakes that can be made in writing a scenario or feature.  Gherkin is a very simple syntactical construct which provides a great deal of flexibility, but as such it also lacks constraints to distinguish what CAN be written as opposed to what is well written.  The constraint must come from the author (or authors) working to provide clarity and removing ambiguity.

It is important to recognize that as feature writers we are all prone to exhibit certain preferences based on our backgrounds and predispositions.  A User Interface expert might put in a description of the interface styling or layout, which is implementation specific.  A tester who is implementing the feature testing in Cucumber might write with a style that would emphasize step reuse or minimization, which might lead to ambiguity at the business level.  A specific customer might drive a feature that is costly to implement but that has no reuse across the wider customer base.  It is the Product Owner who must weigh these competing interests and priorities and establish the common ground with a well written feature.

Compactness is an essential element of a good feature.  Ideally we want just enough verbosity to remove ambiguity, but not so much that the document becomes unreadable.  Remember that the feature file is a document intended for human business-level consumption and legibility, and that the comprehension of the file by a tool like Cucumber is ultimately secondary.

To return to our example, we need to add some information.  There are two kinds of properties involved in the Teddy Bear Conference: business specific and incidental.  The business specific data is, in this case, the Role.  The user and the city and date of a conference are incidental.  The difference is that when writing the feature we need to use fixed terms for Role across the features that involve the term Role (for example always “Attendee, Presenter, Exhibitor”), whereas for city each feature can use essentially any city name, real or invented. If you are using Cucumber for testing it will frequently be the case that have a standard set of these incidental properties will make testing much easier since the fixtures underneath the framework can have very detailed examples for certain cases.  

One of the weaknesses of Gherkin is the lack of a framework to allow a glossary for such business-specific properties as “Role”.  What we do have is a feature level equivalent of “Background”.  So, let us expand our feature with a little more background information.

     Background:

        Given that the valid Roles for attending a conference are “Attendee, Lecturer, Exhibitor”

              And the scheduled Conferences are as follows:

              | city                | date                |

              | Akron           | 12-15-2005    |

              | Austin           | 01-05-2006    |

              | San Diego     | 02-09-2006    |

When this is added to our Basic Attendee scenario, what do we picture on our web page? Maybe a list conferences with check marks that can be clicked, or that can be highlighted or dragged to a visual container of some kind.  For each one that is selected maybe there is a pull down next to it labeled “Role” with “Attendee” selected by default and “Lecturer” and “Exhibitor” available for selection.

Notice that Role is defined as a comma-delimited list and the Conferences are defined in a table.  There is no reason for choosing one over the other so long as the choice is appropriate.  If the Conference had only one parameter instead of the two listed the a comma delimited list might be fine.  Another point on this is that we have fixed dates.  If we want to have all of our Scenarios set in the future we may need relative dates  like “6 months from now” or “Next January 12”. This allows for automated testing that does not need to be updated over time as the relative dates are computed at run time.

Generally speaking if the list of options is one dimensional and fairly limited a comma-delimited list is perfectly fine.  If there are multiple parameters or the text is lengthy, then a table is more appealing.  An additional advantage of a table for lengthy sets is that a table can be used to define a simple alias as in this example for possible warnings or error messages:

        | id | message                                               |

        | 1 | You must select at least one conference|

        | 2 | Two or more conference dates are overlapping- please refine your selection |

        | 3 | Can not select Exhibitor for this conference- all exhibit space is already booked |

It is much easier to maintain and reference an id than the full message text.  Just as good code design leverages code-reuse and enumerations (and databases have look-up tables), good Gherkin has a parallel that drives coding and database design.

Delimited-lists and tables can be combined as well.  This example presupposes a set of conferences that are enumerated A through D:

        | user        | selected conferences        |

        | Bob        | A, B, C                        |

        | Sally        | B,D                                |

        | Tod        | C                                |

Notice that the above table does not include the Role.  There is a good way and a poor way of adding this information”

        | user        | selected conferences and role        |

        | Bob        | A-Attendee, B-Attendee, C-Exhibitor        |

        | Sally        | B-Attendee,D-Lecturer                |

        | Tod        | C-Exhibitor                                |

        

This is not technically wrong as it does not violate the minimalist rules of Gherkin, and for an automated tool like Cucumber, it is relatively easy to handle. It is, however, hard to mentally distinguish the cases involved; it suffers from poor legibility and comprehensibility.  It compacts too much data into a single column, and mixes look-ups with descriptions.  It can be improved in several ways:

        

        | user        | selected conferences        | corresponding roles                        |

        | Bob        | A, B, C                        | Attendee, Attendee, Exhibitor        |

        | Sally        | B,D                                | Attendee, Lecturer                        |

        | Tod        | C                                | Exhibitor                                |

Here we imply an association between the conference and the role by position.  It is still easy to get lost in the data when being read by a human, though, as we need to visually associate a position in a list with another position in another column.  Let us normalize one level further:

        | user        | conference   | role                |

        | Bob        | A                | Attendee        | 

        | Bob        | B                | Attendee        |

        | Bob        | D                | Exhibitor        |

        | Sally       | B                | Attendee        |

        | Sally       | D                | Lecturer        |

        | Tod        | C                | Exhibitor        |

The point of this comparison is to recognize that readability and clarity trumps compactness or step-minimization.  If you are using Cucumber this may result in the need for slightly more elaborate processing of input data or evaluation, but ultimately it will not make it any more complex than is necessary to support the complexity of the product.  It is more important to add a little extra code for a testing step or fixture than to try to shoehorn in more functionality to an existing automated testing code base.  

There are different rules of thumb for indicating different types of data associations.  These can be a challenge to express properly in Gherkin.  While there is no hard-and-fast “best” practice for many of these relationships there are some very clear “don't!” uses of Gherkin.

Let us add a hierarchical event into the scenario.  This is an event that is triggered by some behavior but not others.

Scenario: A user selects Exhibitor as a Role for a Conference.  Capture the name and type of the exhibit.

        Given that the user is at the Conference Registration page

                And the user has selected a Conference city/date combination

        When the user selects the “Exhibitor” Role for the selected Conference

        Then expose the fields for Exhibit Name and Type

                And disable the Submit button

        When the user enters a valid exhibit name and type

        Then the Submit button should be enabled

        When the user submits the page

        Then the user will be registered for that conference

                And the page will navigate to the Completed page

                And will be emailed the attendance info

This is a much more complicated scenario and there are some obvious questions that an implementation will need to answer.  For example HOW will these fields be displayed: as a popup dialog, will visible fields become editable, will fields appear under the selected conference?  All of these are implementation details and are important to a good user experience and are part of the business requirements.  At the level of the feature this is superfluous detail.  What is important is that however it is implemented the user is going to add the necessary data before they can submit their registration.

  

Obviously there are several combinations of statuses that can occur around this scenario.  This suggests that a Scenario Outline is likely more appropriate.  We have implied, but not expressed or described, that the exhibit name and type can be invalid as well as valid, we have implied a possible set of exhibit types, a set of error messages, and a scenario in which the user changes the Role from Exhibitor to something else.

     Scenario Outline:

         Given that the user is at the Conference Registration page

                 And the user has selected a Conference city/date combination

         When the user selects the “Exhibitor” Role for the selected Conference

         Then expose the fields for Exhibit Name and Type

                And disable the Submit button

         When the user exhibit name of <exhibit name> and type of <exhibit type> 

         Then the Submit button should be <button status>

                And <message> should appear

       
     Example:

        | exhibit name   | exhibit type    | button status | message        |

        | Bob’s Bears   | sales              | enabled        |                |

        |                       |                      | disabled        | A valid Exhibit Name and Type is required |

        | Bob’s Bears   |                      | disabled        | A valid Exhibit Type is required        |

        | R                    | sales             | disabled        | The Name must be 3 characters or longer |

        | Bob’s Bears   | whatever       | disabled        | The type “whatever” is not recognized |

There are a number of possible issues with this Scenario Outline.  First, the initial example is identical to the happy path case and would allow the page to be submitted, but because we are mixing it with the other examples we can not include the happy path submittal steps.  This means that this scenario outline can not stand by itself.  The more obvious issue is that for the examples in which no exhibit type is set or the type is not recognized implies that we are allowing a free text entry rather than a set list, and that we have no default.  This suggests either a very, very specific business case or an incompletely considered feature.

So, a quick update to account for these might give us a much simpler scenario.

      Background:

        Given that the valid Exhibit Types are “Sales, Museum, Workshop, Mixed” and Sales is the default

      Scenario Outline:

         Given that the user is at the Conference Registration page

                And the user has selected a Conference city/date combination

         When the user selects the “Exhibitor” Role for the selected Conference

         Then expose the fields for Exhibit Name and Type

                And disable the Submit button

         When the user exhibit name of <exhibit name> and type of <exhibit type>

         Then the Submit button should be <button status>

                And <message> should appear

     Example:

        | exhibit name  | exhibit type    | button status  | message        |

        |                      | Sales             | disabled         | A valid Exhibit Name  is required |

        | RB                | Museum        | disabled         | The Name must be 3 characters or longer |

        | Real Bear      | Museum        | enabled          |                  |

The more constrained the number of permutations the easier the product is to test and to use. The third example can be regarded as superfluous, but we have left it for completeness; it overlaps with the full happy path to submittal.  When thinking through a feature if you find a way to compact the number of scenarios or examples you are likely doing something to improve the usability of the feature and decreasing the likelihood of expanding the feature scenarios to the point where it is hard to mentally digest and understand.  The complexity of your feature in Gherkin is an expression of the complexity of its use, and if you ever opened an iPad or saw an advertisement for it, remember that “you already know how to use it”.  If you never had an Apple product and don’t watch TV, remember the KISS principle- Keep It Simple, Stupid.

Now lets add to our web page a simple Boolean value, in this case “Please send me the annual conference catalog”, and that when this is selected it will change the page to which we navigate when the page is submitted.

So now we have a modification to our simplest case:

Scenario Outline: User registers as Basic Attendee

        Given that the user is at the Conference Registration page

                And the user has selected at least one Conference city/date combination

And has left the default “Attendee” Role for each selected Conference

And selects “<wants catalog>” to receive the Annual Conference Catalog

        When the user submits the page

        Then the user will be registered for the conference(s)

                And the page will navigate to the <destination> page

                And will be emailed the attendance info

Example:

        | send catalog     | destination         |

        | true                  | InputAddress    |

        | false                 | Confirmation     |

There are many ways to express these true/false values in Gherkin.  Using true and false explicitly is one, expressing true and defaulting to false is another (since in most programming languages a Boolean value, when constructed, is set to false), and more elaborate “wants the catalog” and “doesn't want the catalog” is perfectly valid, but only useful when it may not be clear what a true or false actually represents in some case (usually that means the step is not well written so that should set off a red flag).

A few random suggestions

There are some gotchas in Gherkin simply due to the textual format in which it exists.  

• To distinguishing between a null value and an empty string use special text values to distinguish them (“E” for empty vs “NA” of “NULL”)
• Hierarchical data and rules can be difficult to display and contextualize in a workflow; using a look-up table with a parent key for each entry can make this much easier.  
• Use a table to define distinct elements of an object or a page, and to define an enumerated list of the name of the values are very long (as for error messages)
• Use a delimited list to defining a set range of values, such as an enumeration, but not for a disparate set of variables.
• If you have the opportunity, have two people draw who they envision the UI or sketch the underlying object model or database schema that they visualize.  If they are functionally different, try to reword to remove the distinctions.