Why Aren’t All Data Teams Automating Their Tests?
If test automation is so important, why aren’t teams doing it already? In my past life, I was a developer and I learned quickly the value of testing our work. I moved out of development and into project management and business analysis then found my true calling in data warehousing and business intelligence as a consultant. Having worked with several clients around the country, I have seen that data teams are not automating their tests, yet most agree that testing is important. I found this puzzling and so I convened a group of agile data BI consultants in 2015, called ourselves the “Agile BI Lab,” and set out to find an answer. Here it is: The data industry does not focus on testing as much as they do on other disciplines. There are many components that feed into this lack of focus. Following are the top three we discovered in our Agile BI Lab:
1. The Education Isn’t There
There are many places professionals go for education, yet there are few sessions that talk about data testing. In one year, I attended five major events, and there were only two sessions that had the words “testing” or “quality” in them (besides data quality). And I taught one of them. These week-long events hosted thousands of people, and yet there were only two sessions on testing. It is clear that the education on how to test data-focused development is not out there yet. Until recently, if you searched Amazon for a book on “Testing the Data Warehouse” you got nothing. Thankfully, there are now a few books available, so it’s a start, but we have a long way to go on educating our data teams on testing practices and automation.
2. We Don’t Have the Right Team Members
Back when I was working with a very large data warehouse in the ’90s, we were required to go through QA before we could go to production. All that meant was that the BI team would go down to the QA lab and show a QA member that it worked. The QA member always just took our word for it, without really validating anything. It was a huge waste of time and only served the purpose of allowing everyone to check off that box. Our real proof came when we put it in production. We would have working sessions with the key business stakeholders, where we could show the list of business questions that they wanted answers to and see if they could answer them using the BI tool. We actually just walked them through how to answer these questions and that was our final testing, right there in production. This incomplete process is not uncommon in our industry. Simply put, we did not have a good relationship with QA teams because they don’t understand data, and we don’t understand enough about what they do to educate them properly on how to test our work.
3. We Don’t Have Skills & Discipline
I have found that BI teams don’t have many, if any, skills and discipline around QA and testing. Outside of data and BI, I’ve met professionals with really good skills and discipline around testing know-how to put together test plans, scenarios, and cases. They know how to manage test data from the application side and how to make tests that are repeatable and maintainable. They have all of these valuable skills that we, in general, don’t have in our industry.
We Can Change Our Approach to Testing
In Data, we focus so hard and so well on big problems, yet we have veered away from being a part of the testing community. And in doing so, have foregone the immense benefits. It’s time to add solid testing and automation to our pre-production work! We can begin adding testing to our practice by embracing opportunities to learn about QA skills and mindsets. I strongly recommend the Software Quality Association of Denver, or SQuAD, for local folks!
More On Test Automation
Testing drives Agile Analytics. Read my course description: Agile Analytics Testing
Testing the Data Warehouse Practicum. I can’t vouch for this book, yet, but I am providing it as evidence of published works that are becoming available on this subject. Testing the Data Warehouse Practicum: Assuring Data Content, Data Structures and Quality
Test Automation Video featuring Lynn Winterboer and Cher Fox
This video was recorded at the SQL Saturday Colorado Springs meeting held March 25, 2017. The presentation provides a path to how you can bring test automation into your organization’s agile practice and answers:
- Why is test automation important to agile data teams?
- Why aren’t all data teams automating their tests today?
- What is the path to test automation?