June 17, 20268 min read

My AI agent passed CI. My QA gate scored it 0.

Coding agents hand me finished screens that do this all the time. Take a small “recent activity” card for a home view: tsc and ESLint clean, the suite green, the screenshot fine. The kind of diff I almost merge on sight, because a boring, correct-looking diff is the easiest kind to wave through.

Then you score it on five axes, and two come back a hard zero: do not ship.

The screen isn’t broken. It’s wrong while working. The card claims it “synced just now” while the phone is offline, the only marker on the failed row is red text, and a subtitle explains the button sitting right next to it. Nothing throws an error, so every gate that watches for errors stays quiet. A user would feel it in a second: nobody’s home.

The defects that pass every test

This is the failure mode of building with agents in 2026. Google’s 2025 DORA report found AI adoption near-universal and warned that AI amplifies whatever delivery system you already have, making a strong team faster and a shaky one shakier. Faros’s 2025 telemetry put a number on the shaky end: as pull requests ballooned, bugs per developer rose about 9% and time in code review rose about 91%.

But the defect I’m describing doesn’t show up on any of those dashboards, because it isn’t a bug in the usual sense. It’s the code that compiles. An agent writes plausible, test-passing work faster than anyone can review it, and a slice of that work is quietly wrong in ways a test never checks: generic, off-context, or just dead on the screen. Your suite asserts behaviour. It was never going to assert that a screen feels like a person made it. So the defect rides a green pipeline to production, and your users are the ones who find it.

I’d been reviewing the diff. The diff was clean. The screen was the problem, and you can’t see a screen in a patch.

The five-axis QA gate for AI-built screens

The fix that worked for me was to stop arguing about taste and start scoring it. “Feels AI-made” is too vague to act on and too easy to wave through. So I turned it into five axes, each scored 0 to 2, with one rule that does the real work: any single axis at 0 blocks the merge, whatever the total.

Trust. Does the first screen prove it knows the user’s moment? The right state, the right time, no “Good morning” at 11pm, no dangling “3 left” with no unit.
Usefulness. Is the next action obvious, in the app, and where the eye actually lands? Not buried below the fold, not a punt to a search box.
Human-feel. Does every user-facing string sound like a person who understands the context wrote it? One confirmed AI-house-style tell and this axis isn’t a 2.
Native-feel. Is it platform-correct? Safe areas respected, real press states, touch targets at least 44pt on iOS or 48dp on Android, motion under ~300ms, and state that survives the app being backgrounded.
Accessibility. Do labels announce the action and its result, is text contrast at least 4.5:1 (3:1 for large text and UI), is focus visible? In the EU, since 28 June 2025, this has been a legal requirement for a defined set of consumer products and services under the European Accessibility Act, not just good manners.

Add them up. A 10 ships clean. A 7 to 9 ships, with the low-priority items filed as follow-ups. Below 7 doesn’t ship, and neither does any single axis sitting at 0; a 9-out-of-10 with a trust violation is a 0, not a 9. One question rides over all five: would anyone want to reopen this tomorrow? A screen that works but gives nobody a reason to come back is still a miss.

A worked example: scoring an AI-built screen

Here’s that “recent activity” card, scored. The output format matters as much as the verdict: every finding cites a named heuristic, so it isn’t my mood, and the score line shows exactly why it blocked.

The read — it works, and reads like a screen nobody finished.

Findings
1. [BLOCKER] The card says "Synced just now" while the device is
   offline; it's showing the last attempt, not the last success.
   Why: first-screen trust must show the real state, not an
   optimistic one. Fix: show the real last-success time, or
   "Offline · last synced 2h ago".
2. [BLOCKER] The only thing marking the failed row is red text.
   Why: colour alone is not a state (WCAG 1.4.1, Use of Colour) —
   invisible to red-green colour blindness and in high contrast.
   Fix: add an icon or the word "Failed", not just the hue.
3. [HIGH] The row subtitle explains the button beside it ("tap to
   retry the sync"). Why: copy shouldn't narrate visible UI.
   Fix: delete it; the button already says "Retry".
4. [LOW] The card looks tappable but nothing happens on press.
   Why: a control that looks interactive must act, or not look it.
   Fix: drop the press affordance, or wire the tap.

Ship gate — trust 0 · usefulness 1 · human-feel 1 · native-feel 1
             · accessibility 0 → BLOCKER (two axes at 0)

Two zeros on two different axes, and a test runner can see neither. The “synced just now” line is an optimistic lie no assertion in the suite was watching for. The colour-only error state is invisible in a screenshot, and to anyone with red-green colour blindness. Before I scored it, I’d have called the screen “looks fine” and merged. The score is what turns “looks fine” into something you can actually argue with.

The AI tells that fail human-feel

Human-feel is the axis people argue about, so it needs the most concrete list. This is the 2026 default-model house style: what a screen reads and looks like when it was assembled from the current defaults and nobody touched it.

Copy. The em-dash as the only rhythm in otherwise flat sentences. “It’s not just X, it’s Y.” “Let’s dive in.” The relentless rule-of-three. Bold-lead bullets where prose belongs. An emoji opening every bullet. A tidy “in conclusion” that restates the intro.
UI. A violet-to-blue aurora gradient. Frosted glass cards. A bento-grid feature section. Inter or Geist set huge and tight. A “✨ Powered by AI” badge. Everything floating on a soft shadow with a 1px inner border.

None of these is wrong on its own. The tell is the whole bundle arriving at once. And the specific list dates fast; it’ll look quaint in two years, so the part to keep isn’t any single gradient. It’s the meta-pattern: a surface assembled from the current defaults, all at once, with nobody’s fingerprints on it.

There’s a healthy crop of slop-detectors built on exactly this now — design anti-slop skills, “taste” frameworks, the anti-ai-slop blacklists on GitHub. They’re useful and I run one. The gap they leave is the same gap the tests leave. A pattern list can tell you the purple gradient is present; it can’t score the absence of a person, and it doesn’t sit between your branch and main.

Run the gate on your agent’s output

The cheap version is a review pass you hand to a model before a human looks. The wrapper is short:

You are reviewing work against a QA lens (pasted below).

Review this <diff | screenshot | screen>: <paste or attach>
Context: platform = <iOS | Android | web>, surface = <what it is>,
user state = <new | returning | empty | error | …>.

Rules:
- Use ONLY the lens. Every finding cites the heuristic, principle, or
  named tell it comes from. No taste-only opinions.
- Score the five axes (trust, usefulness, human-feel, native-feel,
  accessibility) 0–2 each; any 0 is a BLOCKER.
- Hunt the AI-house-style tells explicitly, in the work AND in your
  own wording. Don't write the slop you're paid to catch.
- Flag anything you couldn't verify from what was given. No invented
  evidence.

<paste the full lens here>

Two honest limits. The wrapper does nothing without a real lens pasted where the last line points; the rules are the cheap part, the criteria are the work. And the model is doing the very thing the gate exists to distrust: producing fluent, confident output that you then have to check. Treat its review the way it treats any agent’s, as a draft to verify rather than a verdict to trust. Agreement isn’t evidence; if you didn’t run the screen yourself, you didn’t review it.

The lens I paste in is mine. I built it the way you build any standard: every defect that reached a user became the rule that would have caught it next time. The fifteen heuristics behind the five axes, the full tell list and code-tell table, and a small slop-check script that runs as an editor hook, a git pre-commit, or a CI gate are the depth in Think Like Roy. The five axes above are the load-bearing part, and you can drop them into a merge gate for agents this afternoon.

What I’d do Monday

Passing CI means the code runs. It never meant the screen was worth shipping, and that gap gets wider the more of the diff an agent writes. You don’t close it with more tests, because this defect is one the tests structurally can’t see. You close it by naming the class and gating on it as seriously as you gate on a red build.

Concretely:

Put the five axes in your PR template, or in your agent’s brief, so the standard is in the prompt before the work starts.
Before you trust a green pipeline, paste the prompt above with a screenshot attached and read the score.
Block the merge on any axis at 0, and judge the rendered screen, not the patch.

Green CI tells you the code runs. Whether the screen is worth shipping is still a judgement, and the five axes are the cheapest way I’ve found to make that judgement out loud instead of in my gut.

Comments & Reactions

Got a thought, a war story, or a “well, actually”? Sign in with GitHub and jump in.

Loading comments…