Shifting Left with Unit Regression Tests
In his book Applied Software Measurement: Global Analysis of Productivity and Quality, Capers Jones highlights that it costs significantly more to repair a defect once organizations release software into production compared to fixing it during the initial coding and unit testing phases. He explains that this is because it takes a lot longer to track down the problem when it is wrapped up with many other modules.
In addition to the cost of tracking down and fixing an issue, there can be a loss of credibility with users as the released software does not perform as expected.
In this blog, we examine the concept of “shift left” and how moving testing to earlier in the software development life cycle (SDLC) using unit tests prevents bugs from slipping through the net into production. We look at the downsides of relying on traditional unit tests and explore how automating their creation helps increase code coverage leading to the release of higher quality software.
The challenge
It used to be the case that developers measured SDLCs in months. This was because they grouped many features to create a single “big bang” launch. It was only after the developers had written the software that the quality assurance (QA) department started testing it to ensure that it matched the required specifications in terms of functionality and reliability. As the software was tested so late in the development cycle, any issues that QA discovered took a long time to isolate and fix due to being entwined with lots of other software modules.
The introduction of agile, continuous delivery and DevOps meant that organizations have significantly reduced their SDLCs with some releasing software features multiple times a day. Continually meeting these velocity requirements means that developers must identify and fix issues early in the development cycle as they quickly move onto their next sprint. Teams cannot afford the luxury of going back to rework something they started multiple sprints ago.
How shifting left helps
The concept of “shift left” involves moving code testing much earlier in the SDLC. In doing so, developers can quickly identify and resolve issues while the code is fresh in their minds rather than waiting for days, weeks or longer before a user finds the problem resulting in them having to relearn the logic, flow and intent of the original code to troubleshoot it.
Shifting left also increases the likelihood that developers will not let issues move through the development lifecycle into the production environment. Issues that make it to production cost significantly more to fix and also decrease the developer team’s or company’s credibility.
Benefits of testing early
The key benefit of testing early or shifting testing left is that developers can identify and resolve issues much quicker than waiting until later in the development lifecycle. This is because developers have not integrated the code modules with other parts of the application making issues easier to isolate.
Due to testing smaller pieces of code, the developers who created the original code can run tests quickly on their laptops. Adopting a “test early, test often” strategy means that developers can catch code issues straight after the code has been written. According to Capers Jones, developers introduce 85% of software issues in the SDLC development phase. Therefore, it makes sense to eliminate them as soon as possible — before they blow out of proportion later when they get released to production.
Challenges using unit tests
We’ve established that shifting left and testing earlier in the SDLC is a good thing to do-but how do we do it in practice?
We have already discussed the idea of testing small pieces of code early using unit tests. So what’s stopping us from using them? The practical implementation of unit tests is hard as traditional unit tests need to be written by developers. A recent Diffblue survey found that developers do not like writing unit tests and 42% admit to having skipped writing unit tests. So, even if our code “passes” the unit test stage, there is a high possibility that we haven’t tested it thoroughly. Lack of thorough testing can lead us into a false sense of security as we believe that we have found all of the issues in our software but in reality only a few basic tests have been passed.
Automating unit test creation
To overcome the difficulty of developers wanting to rightly focus on developing high-value software features and not on writing unit tests, automating unit test creation is the answer. Diffblue Cover’s AI engine examines existing software and derives a set of unit tests that reflect the current functionality. When a developer commits changes to the software, these unit regression tests ensure that the new code has not introduced any unwanted changes. If there are discrepancies between the two versions, Diffblue Cover will alert the developer who can then decide to either accept or reject the alert. If they agree that there has been an unwanted change, they can go back and modify the code to resolve the issue. However, if the developer has made deliberate changes and disagrees that they have made an unwanted modification, then Diffblue Cover will update the tests to reflect this change. When the tests are next run, the tests will reflect the updated functionality accordingly.
Find issues earlier
Shifting left-or testing early-is crucial as it helps developers find and resolve software issues quicker and more cost-effectively before they get deployed into production.
Organizations have traditionally considered unit tests to be the best way for developers to test early and test often. Developers can run unit tests as soon as they have finished writing or modifying a code module. However, it is a better use of a developer’s time to be writing or modifying high-value features to solve user problems in software than to be writing unit tests.
Solving this dilemma involves automating the creation and maintenance of unit tests so that testing can still be carried out early without taking up valuable developer time. The AI engine in Diffblue Cover writes unit regression tests to give developers a quick way to increase code coverage so that regressions can be pinpointed early during the development or modification of a code module.
To learn more about unit regression tests, check out our eBook: “Unit Regression Tests — A new type of test created by Diffblue Cover”
You might also be interested in:
Originally published at https://www.diffblue.com on June 9, 2020.
