Dachary Carey
In our ongoing series about the benefits of git pull request builders, we’ve been looking at the different ways these tools benefit different roles across an organization. Today, we’re looking at the benefits that deploy previews provide for QA teams.
For the QA software professional or QA engineer, the benefits of a git pull request builder include:
With a traditional infrastructure setup, a QA team may have one - or possibly a few - testing servers. In a single-server setup, QA can only test or review a single pull request at a time. When the development team pushes out multiple pull requests per day, but QA can only review one of those at a time, the code review and testing process becomes the bottleneck. This is particularly problematic if you have long-running test suites, or must import large databases to perform testing and code reviews.
In this scenario, the value of a git pull request builder becomes a simple math question; how many testing or code review environments can you have with a pull request builder versus a traditional server infrastructure? The answer is almost always “more for the same cost” or “more for less.” This is also helpful when QA has to request rework due to a bug or regression. The pull request environment can persist, with new code being loaded automatically as the development team pushes an update, instead of having to remove the code from the environment to test another ticket, and then reload it when the rework is complete.
What does the process look like to start testing and reviewing pull requests? QA (or a developer or DevOps team member in some organizations) must pull code into a local environment, or load it onto a testing server. They must load assets they need for testing, which are sometimes large and time-consuming tasks, such as loading a database or large media files. In some cases, they must then kick off an automated test run. Depending on the code base and assets, this could take minutes at best to hours on average; some places have automated test suites so slow that they have to run overnight.
With an on-demand environment automatically created for every pull request, that environment can be building long before QA is ready to test and review the pull request. When a developer is ready to share work, they simply make a pull request, and an on-demand git pull request builder like Tugboat automatically starts building the Preview environment. This could happen hours or even days before QA is ready to test the work, which means all automated tasks can be running against the environment and completed before QA ever starts the review.
With functionality like Tugboat’s Base Previews, and the ability to run online commands once an environment is ready, build time is slashed and automated test runs can be kicked off against the environment once the build is complete, removing manual steps or external scripts that organizations typically need to get this level of CI/CD. Having an environment ready to go when QA is prepared to start testing can dramatically reduce time to start testing and review, and that leads to efficiencies across the QA process and the entire SDLC pipeline.
Common workflows for QA testing include maintaining a local development environment for testing a specific feature branch or changing the local environment for testing every new pull request. With ephemeral on-demand environments for every pull request, there’s no need to maintain a local environment for many simple manual review tasks. And with a git pull request builder like Tugboat, that lets you have shell access into the environment to simplify debugging, you don’t need a local development environment for many QA testing tasks. No local = removing a maintenance and debugging time sink.
The lack of reliance on local environments is particularly helpful when doing QA for migrations. Keeping a database up-to-date on a local environment during a migration, or going back and forth between different states, can be a massive time sink. This is particularly problematic when teams are also doing feature work alongside migrations.
When Digital Services Georgia needed to do migrations and underlying system development simultaneously, Tugboat enabled an interruption-free workflow by running migration code only on migration-related pull requests, and building non-migration Previews without the long-running migration tasks. Take a look at our GovHub case study for more on how Tugboat helped to solve this issue.
Automated testing is one of the best ways for QA teams to gain efficiency and spot regressions, but automated test runs require resources. They need an environment for tests to run against, as well as processing time to run the tests. If you’ve got a single QA environment, or even a handful of them, it’s easy to tie them up with automated test runs while skilled QA folks sit and wait.
With on-demand environments, you can remove automated test runs from traditional infrastructure and run them against the ephemeral environments. Using a cloud-based testing platform that can run tests concurrently, like Testery.io, with an on-demand git pull request builder like Tugboat, can speed up automated test runs by hours.
Beyond the basic functionality of website previews being beneficial for manual QA and automated test environments, Tugboat provides an additional out-of-the-box tool that makes it easy for QA teams to spot visual regressions: Tugboat’s Visual Diffs.
When QA teams define important pages they want to watch for visual regressions, Tugboat takes screenshots of those pages for every new pull request deploy preview, and then compares them to the screenshots in Tugboat’s Base Previews. QA teams can quickly scan visual diffs to spot visual regressions, and then dive deeper into those pages in a manual review to determine what changed, and whether those changes represent bugs or expected changes.
For QA teams that need more advanced visual diffing functionality, such as doing visual diffs for pages behind authentication, Tugboat offers integration with third-party visual diffing tools, such as Diffy.
Today we looked at how git pull request builders benefit QA teams by providing faster time to QA, removing QA environment review bottlenecks, and giving teams additional tools for testing. In our next article in the series, we’ll look at the benefits that deploy preview environments provide for Project and Product Managers. Spoiler alert: it may involve easier collaboration with stakeholders, better visibility into project progress (particularly helpful right now for remote teams!), and increased velocity for teams across the SDLC.
Tell us about your project, schedule a demo, or consult with a Tugboat Technical Account Executive.
