Acceptance Criteria for Data-Focused User Stories: Part 3
In parts 1 and 2 of this blog series, I explained what acceptance criteria are/how they benefit data teams and what happens when the product owner does not accept a story. Now, I am going to walk you through the creation of acceptance criteria.
Creating acceptance criteria is started and led by the product owner, but it involves the entire team to ensure complete understanding and clear guidelines. The team and product owner refine the acceptance criteria until everyone understands exactly what the goals are. More experienced product owners will come to these conversations having anticipated many of the team’s questions and will have them already addressed in their initial acceptance criteria.
Sometimes this can manifest in the opposite manner, with too much detail in the acceptance criteria. Only the most important points should be included as acceptance criteria and the others should be included as notes. It is also important to be able to slice some of those details out and put them into follow-on stories. The idea behind this is that you make it work, and then you make it better later.
Below is an example of exactly what that process should look like. We will use a business intelligence story for this example.
First, the product owner brings the story to the team:
As a call center manager, I want to understand the percentage of total calls that each customer care rep in the company handles so that I can mentor those who are not handling enough.
Next, the product owner provides their initial acceptance criteria:
Results match a manual calculation, based on raw data from both call center applications for the last 12 complete calendar months.
The team now must ask themselves if this is enough information to deliver a working feature. At this point, conversation ensues:
ETL Developer: I wasn’t aware there were two call center applications. I know about the one the customer service representatives use here at HQ – where is the other one?
PO: It’s an old one we don’t use very often, but when customers call in for service-related issues (versus most of our calls are about our products), the representatives log the call in the second application.
Tester: With two sources, it might take longer to load the data each day. What do you want to see during the data load? Do you want to see yesterday’s data until today’s is done loading or show 0% for all representatives until the data is done loading?
PO: I really only need this on a monthly basis for now. Though, I could see a daily view being useful in the future – I’ll make that another user story. I also think I would rather see 0% until the monthly aggregate is completely ready at the end of the month. Let’s add that as new acceptance criteria.
As a result of the above conversation, new acceptance criteria are set:
If the data warehouse does not have a complete month of data, this metric should reflect 0% for all representatives until complete data can be used on the calculation.
The team analyzes these new acceptance criteria and ask themselves again: Is this enough information to deliver a working feature? The conversation could continue like this:
BI Developer: How do you want to see the percentages displayed? A pie chart? A table?
PO: A pie chart with the percentages of each ‘slice’ would be great.
They could then create a lo-fi diagram on the whiteboard:
BI Developer: Don’t we have more than 5 call center reps? Seems like that pie chart would get really busy.
PO: Good point. Could we show the percentages first by teams under each call manager, then drill down to individual representatives?
They could then edit the lo-fi diagram with the added details:
BI Developer: Sure. That’s not hard. Also, when you say you want to know which reps are ‘handling enough’ calls, how do you know what each rep should be handling?
PO: It’s a gut feel for now. I’ll make a new story when the call center managers have found the time to quantify this.
This continues on until the team decides they have enough information to deliver a working feature. These conversations will be different depending on the members, their experience, and how they tend to work. Some need more detail, and others need less.
This team would end with these acceptance criteria:
- Monthly calculation, displaying 0% until the monthly aggregation is complete.
- Results match a manual calculation, based on raw data from both the product and the service call center applications, for the last 12 complete calendar months.
- Pie chart showing the percentages first by the teams who are under each of the call center managers, then drill down to individual representatives.
This looks much different than the original acceptance criteria offered by the product owner. This is completely normal, as the creation of acceptance criteria is a process that involves the entire team.
Creating acceptance criteria is all about gaining a common understanding. It requires the team to think about the proposed acceptance criteria, and then bring their questions and thoughts to the table with the product owner.Creating acceptance criteria is all about gaining a common understanding. It requires the team to think about the proposed acceptance criteria, and then bring their questions and thoughts to the table with the product owner. Click To Tweet