What is shift-left testing?

Have you been involved in a software project that ran over budget and blew past every deadline? Of course, you have–we all have. In fact, if you haven’t, you are a unicorn and I would like to hear from you.

Early stages in my software development career, I learned the importance of working backward from a deadline. If a project must be done by a certain date and testing will take a certain amount of time, then we can use that information to work backward and choose a due date for our project. Perfect, right?

Well, not quite. While building in time for manual testing reduced some stress in the final days of projects, there were still too many surprises.

Building in time for QA testing is great in theory but quickly falls apart in practice once the first bug or defect is identified.

How long will this defect take to fix? How much will it impact the timeline? Will new bugs be introduced? How will we ensure each fix is verified with time to fix anything we broke while we were fixing the first thing?

Ultimately, I was never able to find the correct amount of time to allocate for QA. Inevitably, rushed fixes were merged at the last minute; I learned to keep my calendar clear for a couple of weeks after big launches so that I could triage all of the issues we missed (or introduced) in our mad dashes to the finish.

The problem, at the end of the day, was not the time available for testing but rather the timing of the testing. I needed testing sooner and more often. I needed shift-left testing.

If we imagine our software development process as a timeline flowing from left to right, then “shift-left testing” becomes somewhat self-explanatory. Simply put, it is the practice of testing earlier stages, involving team members, including testers, developers and stakeholders in the testing strategy, and integrating testing for both existing and new features more often in the development life cycle.

The V-model is a useful way to conceptualize software development cycles. If we take the traditional waterfall flow and “flip” the Y-axis at the implementation phase, we get the V-model.

A development cycle begins with high-level requirements. These requirements are narrowed down with each successive step down the “V” until we reach the code-level implementation itself. We then verify the implementation, starting with the most granular unit tests and working our way up the “V” to more abstract user acceptance testing.

In a waterfall process, the entire project is made up of a single “V.” As an industry, we have learned that when you leave all of your validation to the very end of a complex project, you are basically setting yourself up to fail.

In an iterative process, we can think of each sprint or iteration as a smaller “V.” We have theoretically achieved our goals of shift-left: testing sooner and more often. Problem solved, right? Well, not quite.

You may have noticed that there are two labels on the feedback channel built into the V-model: verification and validation. These are both important.

We need to validate that our user requirements actually solve the problems we set out to solve. We also need to verify that our implementation matches the specifications we get from those user requirements.

Automated testing can be applied to both validations and verifications. BDD (Behavior Driven Design) has led to the creation of technologies such as Cucumber that can automate some parts of the validation process. For the purposes of this article, we will focus on automated testing for verification.

Unit tests verify the functionality of a specific module within a larger application. The module is tested in isolation, and any communication with other external processes is simulated or mocked. Unit testing and TDD represent the first development phase in shift-left testing.

Integration tests attempt to verify the overall functionality of a service or application, including side effects. This process is an anti-pattern for reasons we will discuss later.

API tests verify the external endpoints of a single service. The scope of API tests is similar to the scope of integration tests; However, in an SOA or microservices context, we can think of API tests as the new unit tests.

UI tests verify the complete functionality of an application from the user interface layer. Tools like Selenium make automated UI testing widely accessible.

Shift-left isn’t just about automation. Another way to test earlier and more often is to make sure that your QA specialists are involved in every step of your process, beginning with discovery and requirements gathering. Test engineers can do better when they have a greater understanding of the overall implementation, and their insights can help make the architecture more transparent and resilient.

When we think of testing “sooner and more often” a certain word comes to mind: continuous. Many (most) software development teams are practicing some form of continuous integration and continuous delivery. Continuous testing is a vital feedback loop in this DevOps cycle.

If we think of TDD as “shift-left for monoliths,” then continuous testing is “shift-left for distributed architectures.”

TDD had us focus on unit testing. For continuous testing we should focus on API and contract tests. API tests have a number of benefits:

• API tests can prevent one of the most common ways to introduce errors in a microservice application: changing a dependency out of sync with its upstream or downstream services.

• API tests can be owned by the same team that owns the service.

• API tests avoid the brittleness of testing side effects and implementation details.

Ideally, these API tests will be run continuously against both production and pre-production environments. Contract testing tools can help automate this process, but that requires additional infrastructure.

What if we could use continuous API testing built into our observability tool? The upcoming synthetic API testing feature from Instana will let you continuously run API tests against all of your environments with minimal effort.

Shift-left involves moving the testing activities closer to the beginning of the software development lifecycle, enabling faster feedback and reducing the time and effort required for bug fixing. Here are some best practices for shift-left testing in agile development:

1. Early involvement: Testing activities should begin as early as possible in the development process. Testers should be involved from the requirements gathering phase to understand the project scope, objectives and user expectations.

2. Collaboration and communication: Foster close collaboration and communication between developers, testers and other stakeholders. Encourage daily stand-up meetings, sprint planning sessions and regular retrospectives to ensure shared understanding and alignment.

3. Test automation: Invest in test automation to enable frequent and efficient testing. Automated tests should be created alongside the development process and integrated into the continuous integration and deployment pipelines. This helps in detecting defects early, reducing regression issues, and speeding up feedback cycles.

4. Test-driven development (TDD): Encourage the practice of TDD, where developers write test cases before writing the actual code. This shift-left testing approach helps in defining the desired behavior and expected outcomes upfront, leading to more robust and testable code.

5. Continuous integration and continuous delivery (CI/CD): Implement CI/CD pipelines to automate the build, test and deployment processes. This practice ensures that each code change is thoroughly tested and deployed to production environments quickly and frequently, reducing the risk of integration issues.

6. Shift-left security testing: Consider integrating security testing practices early in the development process. Conduct security code reviews, static code analysis and security-focused testing to identify vulnerabilities and address them proactively.

7. Exploratory testing: Alongside automated tests, encourage exploratory testing to explore the application from a user's perspective. Skilled testers can identify potential usability issues, edge cases and scenarios that automated tests might miss.

8. Performance testing: Do performance testing early to identify potential bottlenecks and scalability issues. This helps in optimizing the application's performance and ensuring that it meets the required performance criteria.

9. Test environments and data: Provision test environments that closely resemble production environments to ensure realistic software testing. Also, ensure that sufficient and representative test data is available to simulate real-world scenarios.

10. Continuous learning and improvement: Foster a culture of continuous learning and improvement. Encourage regular retrospectives to reflect on testing processes, identify bottlenecks, and implement changes to enhance the effectiveness of shift-left testing.

By implementing these best practices, agile teams can achieve better collaboration, faster feedback and higher-quality software products through shift-left testing.

The shorter feedback loops built into shift-left processes empower us in several ways. Defects can be found faster, fixes can be applied more efficiently, and lessons learned in one iteration can be applied in the next, to name a few.

Whatever project management methodology or release cadence your team has, you can benefit from the shorter verification feedback loops from shift-left testing.

A defect found by an automated unit test on a developer’s local machine costs less to identify and fix. But when a defect has made it all the way to a customer-facing environment, the cost to fix it increases.

When done properly, automated testing and CI can provide the confidence that software engineers need to deploy often—even on Fridays. Finding defects sooner means fewer panicked all-hands moments. Since releases are so painless, fixing the few errors that do make it through is faster and easier too.

Just as more accessible software is usually easier to use for all of us, more testable software can be easier to reason about and maintain. Thinking about testing early can lead to better separation of concern and a more resilient overall architecture.

Improving the customer experience is our ultimate goal. Shift-left can eliminate some incidents that end users might experience and reduce the impact of other incidents. We can use observability to complete this feedback loop and improve our overall software health.

With powerful automation tools at our disposal, it can be tempting to implement every kind of testing on every line of code. This is a dangerous path.

Testing side effects—did that record actually get saved to the database?—is an attractive idea. But testing implementation details is an anti-pattern because these types of tests are extremely brittle. They might need to be changed every time your application is changed. The user interface is also an implementation detail, so UI tests land in this same boat.

Verification tests just care about the “what,” not the “how” or the “why.” Ideally, the user requirements have been designed to validate “why.” To answer “how”, we can rely on a more powerful automation in the form of an observability platform.

Shift-right testing is the practice of testing later in the development process, usually in production environments. While it may seem strange, shift-left and shift-right testing are complementary.

Shift-right testing allows us to identify production issues before our customers do. The shorter feedback loops from shift-left testing give us the ability to respond to and remediate these production issues rapidly.

Synthetic API testing as a part of your observability platform is the perfect way to combine the benefits of shift-left and shift-right practices.