From widgets to complete applications, the front-end has become a more complex and essential piece in most products’ success, particularly those which include UI and UX in their value proposition.
For this reason, protecting the health of the user experience by ensuring that no regression has been introduced is critical.
While end-to-end tests (E2E) are a complete - but costly - way to assess an application’s health, visual regression testing is a complementary and faster strategy for maintaining product quality. This article will guide you through the many visual regression testing solutions available, from technical libraries to SaaS products.
What is Visual Regression Testing Visual regression tests ensure that an application displays the proper UI in a given user scenario. For example, visit the login page and ensure that all expected visual elements are present (input, buttons, etc.). These visual checks can be performed at two levels:
At the DOM level: by performing snapshots of the output DOM At the screen level: by performing screenshots to verify the actual pixel rendered on the screen An updated version of the well-known pyramid of tests would now be the following:
Unlike E2E tests, visual regression tests do not necessarily require interacting with the page, making them faster to run and easier to maintain.
How to Use Visual Regression Tests Both types of visual regression testing (snapshot and screen) should be performed against the critical path of the application. An application’s critical path is the set of pages and features that define your product experience's core, for example, a login page, a register page, or a cart checkout page.
Snapshot Tests Snapshot tests, mainly performed with Jest, are commonly considered part of integration tests. They consist of rendering a component in a given scenario, and saving a snapshot of the resulting DOM. When the test gets updated, the new DOM result gets compared with the existing one.
If any change is found, an error is raised to notify of the regression.
Because snapshots tests rely on DOM comparison, they should test the structure of a given page or set of components.
For example, snapshots ensure that:
A set of actions is available on a page (ex: filtering, searching) The proper information text is displayed The login page does not submit the form if the inputs are empty In short, snapshots tests are perfect in preventing any visual regression in the structure of your application (actions, text, etc).
While snapshots tests are easier to set up and maintain, and fast to run (< 2s), they cannot be used to assess pure visual aspects such as CSS overflow issues or colors of UI elements. You can read more about how to implement snapshot tests here .
Screen Tests Screen tests give us a finer visual regression detecting by comparing screenshots:
With a bit more setup work, Screen tests prevent any kind of visual regressions, from structure (action buttons, text) to styling (overflow, colors, padding).
Testing advanced interactions or image loading scenarios is possible, thanks to tools like Cypress, which rely on a virtual test browser. However, running a virtual test browser comes with its drawbacks: a more complex CI setup and slower tests (>1min).
For this reason, screen tests should be used to assess the final rendering of a given page part of the critical path or UI-Kit components. For example, ensuring that the onboarding pages are rendered correctly or that all components of a UI-Kit are correctly rendered. You can read more about how to set up screen tests here .
SaaS Solutions The previously mentioned technical solutions fit well for small teams where only technical users need to access the results of the visual tests.
SaaS solutions exist for bigger companies or ones where product/designers are in the visual testing process, such as Meticulous , AppliTools and Percy .
Covering the features offered by the major visual regression SaaS solutions would require an article on its own. Still, most SaaS solutions offer the following features that ease the setup of advanced visual regressions tests and enhance the collaboration on failing tests:
Page and component testing Dedicated cloud worker for faster execution A platform for reviewing regressions (for technical and non-technical people) Advanced Workflow management AI features to avoid false-positive results Workflow integration: Github, Jenkins, Slack, and more Conclusion Front-end is now a critical part of any web-product-based business.
While maintaining the quality of their products by implementing many E2E tests, we see that visual regression testing is a faster solution.
Snapshots tests as integration tests are helpful to ensure that the visual structure of a page is maintained over changes. Screen tests are slower to run and should avoid any visual regression in essential components of an application, for example UI-Kits.
Finally, depending on the product complexity and the size of your organization, choosing SaaS solutions to enhance collaboration and set up more advanced tests might be the way to go.
Meticulous Meticulous is a tool for software engineers to catch visual regressions in web applications without writing or maintaining UI tests.
Inject the Meticulous snippet onto production or staging and dev environments. This snippet records user sessions by collecting clickstream and network data. When you post a pull request, Meticulous selects a subset of recorded sessions which are relevant and simulates these against the frontend of your application. Meticulous takes screenshots at key points and detects any visual differences. It posts those diffs in a comment for you to inspect in a few seconds. Meticulous automatically updates the baseline images after you merge your PR. This eliminates the setup and maintenance burden of UI testing.
Meticulous isolates the frontend code by mocking out all network calls, using the previously recorded network responses. This means Meticulous never causes side effects and you don’t need a staging environment.
Learn more here .
