Test Strategy for Continuous Delivery::Shifting Left & Shifting Right
monica914 060000WAY8 Visits (21531)
A couple of months ago at our Interconnect conference, I was on a panel that led to an interesting discussion with a customer. The panel was on “Shift Left Testing”, but I had also introduced the idea of moving some testing to production. The customer said to me, “so some testing is shift-left & some is shift-right.” For me it was an “aha” moment crystallizing things I’ve been thinking a lot about and I’m co-opting the term.
Indeed, that’s exactly what we need to do when considering our test strategy for Continuous Delivery. I’ve previously written blogs on the topic of how we think about and build our test automation for Continuous Delivery. And also how we need to think about quality as being in the eye of the beholder. My thinking continues to evolve on this topic as I talk with customers and internal IBM teams. For instance, how do we “map” our testing concepts of UAT, SVT and Performance testing to Continuous Delivery of as-a-Service offerings? My answer is: we don’t!
Getting back to basics and the old saw that “you can’t test quality in,” we need to consider quality not testing. I’ve recently been involved in a couple of Code Retreat events including as a participant. Even as an old-hand at automated testing, it was a revelation to do test driven development (TDD). Of course, one does some upfront design & thinking about classes & methods to achieve an outcome. But as you start to implement, writing the test first really influences the actual code you write. It’s better, clearer and, heavens, more testable. When tests are written after (even when the test writer is the same programmer), there are testing limitations. The code was not written with “how am I going to test this?” in mind. Universally around the room during the Code Retreat reflection, the engineers said “I had no idea how different this is.”
TDD is the ultimate in Shift-Left testing.
And so is Behavior Driven Development (BDD) which has been popularized by Cucumber (although there are many, many ways to implement BDD). BDD is an opportunity for the Product Owner (or maybe it’s a PLM in your organization) to align around the business value of the capability. The user story describes the desired outcome, but the BDD scenario describes the acceptance test. Writing the scenario before writing the code changes the implementation because it’s being done in the context of the desired outcome. BDD is an awesome technique that ties back to the quality in the eye of the beholder or in “lean speak” eliminating uncertainty about what you are building. It also helps to establish the minimal viable product (MVP). A good user story and scenario would include the metric to measure to understand whether your customer wants what you are building. Automating the BDD requires setting up integrated environments and integration tests/testing are a natural outgrowth. Whenever possible, automate these scenarios below the GUI to ease test code maintenance.
In the case of both TDD & BDD, the good news is that teams typically build that into the CD pipeline first, so every test written is automatically (read through unattended automation) run all the time.
What about the test strategy for those big, hairy things like Mean
Everything adds up to creating the right balance of testing the capability in-market to ensure we are solving the right problem and in the right way, along with new strategies for validating functionality and platform excellence. In order to do all of these things at the speed our business demands the entire team needs to be rethinking our approach to ensuring quality that understands AND leverages the Cloud.