A Screenshot Cannot Test the Interaction That Produced It
A modal can look perfect in a screenshot while being impossible to close with Escape, leaving keyboard focus behind the overlay, or submitting a form twice when a slow network response arrives.
That is not a criticism of visual regression testing. It is a reminder to ask the image the question it can answer. A screenshot compares pixels at one settled state. It does not prove how that state was reached or whether the next user action is valid.
Pair assertions with the risk
For a design-system change, a screenshot is excellent evidence that spacing, colors, and responsive layout did not drift. For a checkout flow, add a behavioral test that exercises the transition and asserts the observable result.
await page.getByRole('button', { name: 'Edit billing address' }).click();
await expect(page.getByRole('dialog')).toBeVisible();
await page.keyboard.press('Escape');
await expect(page.getByRole('dialog')).toBeHidden();
await expect(page.getByRole('button', { name: 'Edit billing address' })).toBeFocused();
This test has no visual assertion. It verifies the keyboard contract: open, close, and return focus. A screenshot of the open dialog would not reveal any of those facts.
The distinction matters beyond accessibility. A cached fixture can produce the right visual state even when an API request used the wrong authorization header. A test that starts from a preloaded URL can hide a broken navigation transition. A hydration race can disappear by the time a screenshot is captured.
Keep visual tests narrow
Screenshot tests become noisy when they absorb dynamic clocks, random content, remote ads, or loading animations. Stabilize known nondeterminism, scope the capture to the component under review, and use an intentionally reviewed baseline. Do not respond to visual flakiness by removing browser behavior tests; they are testing different failure modes.
References
- Playwright visual comparisons describes screenshot-based regression testing and its baseline model.
- WAI-ARIA Authoring Practices: Dialog modal pattern specifies keyboard and focus behavior that pixels cannot show.
Final take
An image is evidence of appearance, not interaction. Test both when both matter.
Changelog
- 2026-10-03T00:08:49-04:00: Recovered the 2026-07-16 scheduled slot after archive comparison and primary-source review.