AI Visual Testing: Set Match Rules and Review Baselines Before Approval

A checkout screenshot stops failing after the team relaxes its comparison rules. That could mean a changing date no longer creates unnecessary alerts. It could also mean the payment total, button wording, or warning color is no longer being checked. A quieter report is useful only when reviewers can explain which differences became acceptable and which defects the test should still expose.

AI visual testing compares rendered checkpoints with reference images using a chosen comparison policy. In baseline-based visual regression testing, the reference represents the expected appearance for a defined context. This differs from an agent using screenshots to navigate an application without a saved reference. Neither approach makes a screenshot sufficient evidence that a payment calculation or submission behavior is correct.

Fix the Checkpoint and Reference Before Tuning Sensitivity

Start with an existing automated UI journey that reaches a meaningful state, such as a completed order summary before payment. Identify what must be visible and which application state makes the image valid. Prefer controlled test data and an observable readiness condition over capturing whatever happens to appear after a fixed delay. If the screenshot catches a loading placeholder, changing match sensitivity treats the wrong problem.

Record the application and test identity, checkpoint sequence, browser, operating system, viewport, and relevant branch. Also identify the baseline and its revision when reviewing a result. These details let another reviewer determine whether the comparison used the intended reference rather than a different environment or an older design.

In Applitools Eyes, baselines are associated with test and execution-environment attributes, and selection can use policy-dependent version compatibility and fallback. Do not assume every browser-version change creates a separate reference, or that a familiar screenshot necessarily came from the baseline you intended.

Review the first reference deliberately. Current Eyes documentation describes first-run checkpoints being added and accepted automatically as baseline images. That initializes a comparison history; it does not establish that the application implements the approved design. A team can require a person to inspect those images before relying on the suite. That is a recommended team gate, not a claim that the provider requires human approval before creating an initial baseline.

Choose Match Rules for the Requirement

Choose the comparison policy by asking what the check must preserve. A requirement about button wording needs content verification. A requirement about warning color needs color verification. A requirement about a recommendation grid may tolerate different products while retaining expectations about structure. Applying the same permissive policy to all three can remove the evidence the test was meant to supply.

Compare What Each Policy Retains

The following distinctions use current Applitools terminology. Match levels and region controls serve different purposes, so treat the last two rows as region-specific tools rather than interchangeable global sensitivity settings.

Policy or Region	Appropriate Variability	Coverage Boundary
Strict	Mostly stable content with tolerance for rendering differences	Checks text, font, color, graphics, and position; still requires review against the requirement
Layout	Content varies while relative structure should remain consistent	Ignores actual text, graphics, color, and other style differences
Ignore Colors	Content remains stable but color variation is acceptable	Color changes no longer provide a failure signal
Dynamic	Defined text patterns, such as dates, can change	Valid pattern matching does not establish the correct business value
Ignore Region	A specific area is deliberately excluded as a last resort	Visual AI comparison does not occur inside that area
Floating Region	An element may move within an explicitly allowed range	Permits movement within the configured bounds; those bounds need a requirement

Eyes also documents Exact, a pixel-to-pixel comparison that is sensitive to rendering anomalies and is not recommended for ordinary verification. More sensitivity is not automatically more useful evidence. The appropriate policy distinguishes unacceptable change from variability the team has consciously allowed.

Check the installed SDK and effective configuration instead of assuming a universal default. The current match-level documentation makes Dynamic the default for Web tests on supported SDK versions, with minimum-version requirements and DOM capture enabled by default to support it. For example, its JavaScript list starts at eyes-playwright 1.47.5 and eyes-cypress 3.60.3. Those thresholds are specific to those packages, not a statement about every SDK. Dynamic matching is documented as Web-only; do not extend that behavior to native mobile tests. Explicit configuration and team settings also need inspection.

Keep Critical Values and Layout Under Deliberate Checks

A date can match an allowed format and still promise the wrong delivery day. A currency value can have valid formatting while showing the wrong total. If the actual value matters, retain a separate assertion for that requirement rather than treating Dynamic matching as proof of correctness.

Position needs the same care. Current documentation says that full-checkpoint Dynamic matching automatically tolerates surrounding positional shifts caused by dynamic content or regions. That may be useful for a growing content area, but it can remove a displacement signal your design requirement needs. For a checkout with critical copy, color, and placement expectations, explicitly choosing Strict for the checkpoint and narrowing permitted variability to selected regions is a defensible starting policy. Validate the resulting coverage rather than assuming any policy detects every overlap or rendering defect.

Scope Dynamic Regions Without Hiding the Checkout

Suppose an illustrative checkout contains a dispatch date, a recommendation panel, the payable total, and a Pay button. The date changes between runs; the total and payment controls must remain correct. Define the smallest useful area around the changing date, not a rectangle spanning the entire order summary.

Region selection and match behavior are separate decisions. In Eyes, an element-based region can follow the relevant web element when its location changes, but requires DOM capture. A drawn region uses screen coordinates. Neither guarantees that the selected area matches your intended business boundary. Inspect the actual element or rectangle, any padding, and its behavior at the viewports you test. Padding added for a shadow can also pull a neighboring label into a more permissive comparison.

Use the least permissive treatment that fits the variability. Layout may suit a recommendation panel whose products change but whose arrangement matters. It also ignores product text, artwork differences, and styling, so it cannot establish that the correct recommendation or brand color appeared. An Ignore region goes further: the excluded area is not visually compared. It is not a substitute for checking the panel's layout.

Reserve exclusion for content whose lost coverage is acceptable, such as a nonessential third-party creative in this fictional flow. Keep the total, payment label, and critical warning outside it. If placement of the excluded content's container matters, retain a separate check for that requirement. Do not interpret an ignored area as verified, and do not use a large mask to make an unexplained failure disappear.

Keep a short reviewable record for each exception:

• The exact element or area and the variation allowed there.
• The content, color, or placement checks that the policy removes.
• Any separate assertion needed for a critical value or neighboring component.
• The owner and the design or responsive-layout change that triggers another review.

This record is a suggested team convention, not a set of built-in provider fields. Its purpose is to make a future reviewer understand why the exception exists and what would make it unsafe.

Review Differences Before Saving a New Baseline

A baseline approval workflow needs two independent questions: did the UI change as intended, and should this captured image become the reference for future comparisons? Replacing a baseline answers the second question mechanically. It cannot establish the first.

Use a review sequence tied to the actual change:

1. Confirm the tested revision, environment, branch, checkpoint sequence, and reference identity. Check that the images represent the intended completed application state.
2. Inspect baseline and checkpoint images together, including critical areas outside the highlighted differences. Review added and missing checkpoints as well as changed images.
3. Compare the result with the requirement or approved design. Look for expected changes that never appeared, even when the screenshot matches its old baseline.
4. Classify each outcome as an intended change, a product defect, a test or data problem, or a permitted variation requiring a narrowly justified rule change. Preserve the original evidence and reason for the decision.
5. Inspect the maintenance scope before accepting, rejecting, or changing regions. Start with the current step when assessing an unfamiliar change; examine the affected set before applying an operation more broadly.
6. Save only the intended reference changes, record their relationship to the reviewed commit or pull request, and inspect subsequent targeted results under the chosen policy.

The scope check is important in Eyes because accept and reject operations participate in Automated Maintenance. Depending on its configured scope, an action can affect other steps, and the editor may prompt you to apply maintenance elsewhere. Similar-looking differences do not automatically share the same product intent. A changed label on checkout and a changed label on account cancellation can require different decisions.

Eyes documents accepting an expected changed checkpoint as replacing the obsolete baseline image. Rejecting a defective difference keeps the baseline image and marks the step as failed. Review changes are stored automatically, but applying baseline changes to future runs requires saving them to the baseline. A review decision and the saved reference are therefore related but distinct states.

The documentation also permits saving while some steps remain unresolved. If your release process requires all critical visual questions to be resolved first, implement and verify that rule in your own workflow or CI gate. Do not assume that the ability to save, a green result, or a completed review action enforces it. Require evidence from the current tested revision and a named reviewer for intentional reference changes; define how a missing report or unreviewed change blocks the decision. These are proposed local controls, not built-in guarantees.

Resolve a Checkout Change Without Bulk Acceptance

Consider a fictional change request that replaces “Pay” with “Pay Securely” and adds a delivery message. The new run also contains a changing dispatch date and an unexpected overlap between the total and a promotional banner. These are different review decisions, even if they appear in one screenshot.

The wording change may be suitable for baseline acceptance after checking the approved text, its placement, and nearby controls. The overlap should remain a defect, not become part of the new reference because the label change was intentional. Narrowing a region around the date may remove permitted date variation, but it should not weaken comparison of the total or banner boundary. If the promised delivery date itself matters, its expected value still needs a separate check.

Now suppose the delivery-message checkpoint matches the old reference because the planned message was never implemented. There may be no visual difference to approve. The reviewer must still reject the result as evidence of the requested change and investigate the missing behavior. Current Eyes guidance explicitly recognizes that a matching step can represent a bug when an expected new feature is absent.

A known issue needs an owner and an explicit risk decision under the team's release policy. Saving its appearance into a baseline does not resolve it. Keep the issue and any temporary exception visible, including which evidence is missing and what condition ends the exception. Avoid making “matches the reference” synonymous with “approved for release.”

Prove the Review Policy in a Small Pilot

Before expanding the suite, propose a bounded nonproduction exercise on the checkout flow. Deliberately alter critical wording, a required color, and a meaningful layout position, then inspect whether the selected policies produce actionable evidence. Also check an allowed date change, a missing expected component, and a report that is absent or lacks the required review. These are validation tasks to perform, not results established by this example.

Keep functional assertions for totals and submission behavior, and appropriate accessibility checks for requirements such as keyboard operation. Screenshot comparison adds appearance evidence; it does not establish those outcomes on its own. For a broader framework for choosing pilot scope, owners, and evidence gates, use this test automation strategy guide.

Expand only when reviewers can explain the allowed variability, identify the checks that exceptions remove, and demonstrate that important controlled defects still receive the intended response. A smaller suite with explicit comparison and approval boundaries is a better basis for the next investment than a quiet dashboard whose lost coverage nobody can describe.