Reprinted with permission from Media 1105, Inc. Originally published September 12, 2017 – TDWI Flashpoint Article
Problem: One of the biggest challenges for agile data teams is embracing test automation as a technical enabler for agility.
Fellow data professionals often ask me for recommendations for test automation tools, thinking that the implementation of a tool will make their process easier. My advice is to defer focus on a tool until the team can consistently define useful, repeatable tests that can be automated.
An easy experiment your data team can try is to work with a friendly business stakeholder to define a small improvement (i.e., one that would make a person’s work life better) to an existing report, then define together the repeatable and automatable business-facing tests that would prove the work has been done correctly.
The method I prefer to use is called behavior-driven development (BDD), which has at its foundation the use of a domain-specific common language the customer uses to describe needs and wants.
This means that all requirements, models, and deliverables use the natural language of the customer, not the technologies used to deliver the value.
Because the common language underlying BDD enables a team to focus on testing earlier in the process, each piece of work is described in terms of what the business needs and how it will know when a request is completed. These types of tests—written before any design or development work begins—are called “executable specifications.” Before I describe why BDD tests are written the way they are, consider the following user story:
As a sales manager, I want to know that when a sales rep moves from one sales region to another, the resulting sales reporting will include that change, so that I can stop manually recalculating certain sales reports to accurately reflect where the rep was assigned at the date of each successful sale.
There are many acceptance tests defined in collaboration with the business. For instance, a team may be asked to confirm that the orders sold by June Jones, who switched from District A to District B on Dec. 1 of last year, are properly allocated to the right district per month.
The resulting executable specification written in the BDD format is:
Given June Jones is a sales rep in District A
And she has sold $150,000 in Q4 as of Nov. 30,
When she switches to District B starting on Dec. 1
And sells $50,000 in that month,
Then the BI sales report should show:
As you can see, there are some keywords that are important to BDD:
- Given is the precondition to the test and establishes the initial state of the data
- When is used to define an action that changes the state of the data
- Then shows the expected outcome after the action has occurred
- And describes a continuation of Given, When, or Then
- But can be added to identify a negative condition (opposite of And)
There are other keywords in this syntax, but these are the most useful.
The test is specified before any data modeling, development, or testing has begun. Accordingly, this means the specification for what needs to be delivered must be clearly written so that the delivery team can efficiently execute on it and confidently arrive at accurate results. The specification provides an unmistakable example of what the customer (sales) expects to see in a specific situation. All other similar situations should follow the same pattern—or be clarified in other executable specifications. Finally, with defined inputs and clear expected outputs, this test is now repeatable and automatable!