Tugboat’s visual regression testing feature, visual diffs, is an important part of the workflow for many of our clients. Not only is visual regression testing is additional line of defense against bugs making it through to production, but providing this functionality also enables teams to develop and deploy faster, with higher confidence that each new pull request isn’t introducing breaking changes. With that in mind, we’ve super-charged this functionality to make visual diffs even more powerful.
Do you have pages where content changes regularly? Maybe you’ve got an ad that displays different images, or a slide-in feature product or content carousel that may display something different based on personalization. In those cases, you’d expect to see some difference in the page, so a 100% match in visual diff screenshots isn’t a requirement.
With visual diff thresholds, you can tell Tugboat to set a pass/fail status based on how similar your pull request Preview (the After screenshot) is to your Base Preview (the Before screenshot). If you’re A/B testing different headers, you might only expect 90% similarity between pages. If you’ve got a large image or section that might change, you may only need a 70% similarity between images. You can define these thresholds on a per-URL basis, and even a per-breakpoint basis, to make your visual regression testing more meaningful. This way, you’ll know that a failed visual diff is really a fail - not just an expected change to your page.
When you use visual diff thresholds, you get the added benefit of communicating more effectively with developers in the places they work - by automating visual regression tests against pull requests in git repositories. With thresholds enabled, Tugboat only sends a “Pass” status to GitHub, GitLab, or Bitbucket if your visual diff meets or exceeds your similarity threshold. If it doesn’t, Tugboat sends a “Fail” status to the linked git provider - and the pull request can’t be merged without an override.
This added step helps communicate to team members to check the visual regression tests and fix issues that are causing the pull request to fail. Once these visual issues are corrected - or someone reviews and approves the visual diff - the status changes to “Pass” and the pull request can be merged. This status reporting makes it easy to add a visual regression review process to your workflow.
What happens when you see a “Fail” status on a Tugboat visual regression test? Click into the Details to open the Tugboat Dashboard and view all of the visual diffs for the pull request. This at-a-glance-view makes it easy to spot visual diffs that aren’t passing the threshold. Team members can click into Desktop, Tablet, or Mobile breakpoints, review the visual diffs, and manually change the status to “Approve” if the changes are expected and the pull request shouldn’t be blocked.
Alternatively, reviewers can manually “Decline” a visual diff that meets the threshold if they spot an issue during the review process.
Make visual regression testing and review an easy part of your workflow with these new features, and enjoy higher confidence in deploying bug-free code to production.
Tell us about your project, schedule a demo, or consult with a Tugboat Technical Account Executive.