Most writing about AI and visual testing focuses on one thing: cutting down false positives in pixel diffing. That's a real problem, but it's not the interesting one. Pixel diffing can only catch a bug if the baseline was correct to begin with. If a button was already overlapping text when the baseline got captured, or if you're testing a screen that has no baseline yet, diffing has nothing to compare against and tells you nothing. It's built to detect change, not to judge whether what's on screen is actually correct.

A different use of AI: semantic assertions, no baseline required

Instead of comparing screenshot A to screenshot B, you can ask a vision-language model a targeted yes/no question about a single screenshot, independent of any baseline history. This turns visual QA into something closer to a real assertion. It doesn't ask "did this change," it asks "is this correct."

That shift matters because it catches an entire class of bugs pixel diffing structurally cannot see: overlapping text, truncated buttons, broken image placeholders, a modal misaligned on a page you've never tested before. None of these require a previous screenshot to identify. A human looking at the page for the first time would spot them immediately, and that's exactly the judgment a vision-language model can approximate.

This isn't meant to replace your regression suite. It's a second, independent check that doesn't care whether anything changed, only whether what's currently on screen looks right.

Code example: baseline-free visual assertion

Here's what that looks like in practice. The function takes a screenshot and a plain-language question, sends both to a vision-capable model, and returns a boolean based on the answer.

                
import anthropic, base64

client = anthropic.Anthropic()

def visual_assertion(screenshot_path: str, question: str) -> bool:
    with open(screenshot_path, "rb") as f:
        img_b64 = base64.b64encode(f.read()).decode()

    response = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=10,
        thinking={"type": "disabled"},
        messages=[{
            "role": "user",
            "content": [
                {"type": "image", "source": {
                    "type": "base64", "media_type": "image/png", "data": img_b64
                }},
                {"type": "text", "text": f"{question} Answer only 'yes' or 'no'."}
            ]
        }]
    )
    return response.content[0].text.strip().lower().startswith("yes")
                

You drop this into an existing suite as a secondary check on the screens where correctness matters most, not on every component snapshot in the suite.

                
# In a test
assert not visual_assertion(
    "checkout.png",
    "Is any button text cut off or overlapping another element?"
)
                

No baseline file, no prior run, no diffing library involved. The assertion stands on its own, which is exactly what makes it useful on a screen you're testing for the first time.

Where this breaks down

Non-determinism. Ask the same model the same yes/no question about the same image twice, and you won't always get the same answer, especially on borderline cases like a button that's almost, but not quite, overlapping a label. Forcing an immediate binary token means the model has to get a spatial judgment right in a single pass, with no room to reason about it first.

The fix is simple: let the model think before it answers. Asking it to describe the relevant layout in a sentence or two before returning a verdict measurably improves accuracy on exactly this kind of spatial judgment call.

                
def visual_assertion(screenshot_path: str, question: str) -> bool:
    with open(screenshot_path, "rb") as f:
        img_b64 = base64.b64encode(f.read()).decode()

    response = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=150,
        thinking={"type": "disabled"},
        messages=[{
            "role": "user",
            "content": [
                {"type": "image", "source": {
                    "type": "base64", "media_type": "image/png", "data": img_b64
                }},
                {"type": "text", "text":
                    f"{question} First, briefly describe the relevant layout "
                    "in 1-2 sentences. Then end your response with exactly "
                    "one line: 'FINAL: yes' or 'FINAL: no'."}
            ]
        }]
    )
    answer_line = response.content[0].text.strip().splitlines()[-1]
    return answer_line.lower().endswith("yes")
                

Two things change from the original snippet: max_tokens goes up to make room for the reasoning, and parsing shifts from "does the whole response start with yes" to "does the last line end with yes." That tradeoff, a slightly slower and more expensive call in exchange for a more reliable verdict, is worth making on any screen where a false negative actually matters.

Cost and latency. Even with a short reasoning step, this is still an API call per screenshot per question. Firing it on every assertion across a large suite adds up fast in both time and spend. Reserve it for the screens where correctness genuinely matters, checkout, signup, payment confirmation, not as a blanket check on every component snapshot.

Not a diffing replacement. This catches absolute defects on a single screenshot, but it says nothing about drift over time. A page that's slowly getting worse release after release, spacing creeping a few pixels each time, contrast fading gradually, is exactly what pixel or perceptual diffing is built to catch and what a single-screenshot question has no way to notice. The two approaches are answering different questions, not competing for the same job.

Closing: use both, deliberately

Pixel and perceptual diffing are still the right tool for catching drift over time, the slow, cumulative regressions that only show up when you compare today's screenshot against last month's. Nothing in this post is meant to replace that.

What it adds is a second, independent layer built for a different question: not "did this change," but "is this correct, right now, with or without a history to compare against." Run diffing across your suite as usual. Layer semantic assertions on top of it for the handful of screens where a wrong answer actually costs something, the ones where a broken checkout button matters more than a shifted pixel a diff tool would flag anyway.

Diffing tells you something changed. It was never going to tell you whether what's on screen is right. That's a different job, and it needs a different tool.

A working version of the code in this post, including the reasoning-before-verdict fix and a runnable demo, is available on our GitHub page.