What is verification engineering?

Verification engineering is the practice of building the checks that prove AI-written code works before it ships. Instead of reading every line an agent writes, engineers design the tests, environments, evidence and approval steps that tell agents and reviewers whether a change is correct.

The term is new. It's spreading because coding agents now write code faster than people can review it, so the hard part of shipping has moved from writing code to trusting it. This guide covers what verification engineering means, why it matters now, the principles behind it, and how to start.

A note on the name: in chip design, "verification engineer" usually means a hardware design verification role that tests silicon with tools like SystemVerilog and UVM. This article is about verifying software written by AI coding agents.

What is verification engineering?

Engineer Andrew Allbright defines it as "the practice of discovering, designing and integrating deterministic feedback that help make software correctness observable and actionable." In his words, the verification system works like a set of sensors for the agent, giving it an outside signal about whether what it wrote is wrong.

Ryan Fitzgerald puts it more simply: "building the right systems and checks to catch an agent before it produces garbage."

Both point to the same shift. The care engineers used to put into writing each function now goes into the systems that judge the code. The output of verification engineering isn't the feature itself. It's the confidence that the feature does what it should.

Why verification is the new bottleneck

When people wrote most of the code, writing was the slow part. Now one engineer can run several agents at once, and each one comes back with a pull request.

OpenAI's harness engineering report shows the scale. Over five months, a team that started with three engineers driving Codex opened and merged roughly 1,500 pull requests, building a repository of around a million lines of code with no manually written code. At that pace, nobody can read every line.

Engineering leader Jade Rubick makes the same point in an Aviator interview: "Engineering teams need to be thinking about how they can get out of verifying every line of code, because otherwise they're going to be completely buried in that and will absolutely be the bottleneck."

The risk runs both ways. If agents write code faster than anyone checks it, bugs ship faster too. Verification engineering is how teams keep the speed without losing control of quality.

Testing vs verification vs validation

These words get used loosely. Alex Lavaee, who worked on coding agents for Microsoft's Windows team, separates them in a conversation with The Neuron:

	The question it answers	Example
Testing	Does this behavior pass the checks we built?	A unit test confirms the login button responds to a click
Verification	Does the change meet the agreed criteria?	The user can log in and gets a working session
Validation	Do those criteria solve the real problem?	Customers can actually get into their account without support tickets

His example: a login button can respond to clicks and pass a narrow test while never creating an authenticated session. The test passed. The product failed. Verification engineering sits in the middle, connecting what was asked for to evidence that it was delivered, while people stay responsible for whether the ask was right.

Principles of verification engineering

Lavaee lists six principles in the same discussion:

1. Specify objective success criteria before the work. Decide what "done" looks like before an agent writes anything.
2. Separate author and judge. Don't let the agent grade its own homework.
3. Prefer deterministic checks before subjective model judgment. A passing test beats an AI saying something works.
4. Verify in a running environment. Test the real application and its important dependencies, not just isolated functions.
5. Save evidence of execution. Keep screenshots, logs, recordings and test results.
6. Require human approval for criteria and consequential steps. People decide what success means and approve what matters.

He adds a practical rule: give riskier changes more verification. A copy fix and a database migration shouldn't get the same treatment.

What a verification system includes

Most teams build verification in layers, from fastest and cheapest to slowest and most expensive.

• Static checks. Type checks, linters and formatters catch whole classes of mistakes in seconds.
• Automated tests. Unit and integration tests confirm behavior the team has already defined.
• Checks in a running app. End-to-end tests and browser checks run against the real application in an isolated environment.
• Visual evidence. Screenshots and recordings show reviewers what changed without making them check out the branch.
• Independent review. Fitzgerald recommends having a different model review the agent's work rather than the same model in another thread.
• Human approval. A person with merge rights reviews the evidence and decides what ships.

The fast layers should run constantly while the agent works, so it can fix its own mistakes. The slower layers run before a person ever looks at the change.

Verification engineering vs QA

Traditional QA often sits downstream of engineering: developers finish, then hand work over for testing. Rubick describes a different role he calls the automated verification engineer, one "much more focused on creating fast feedback for agents and humans."

In that model, verification is the whole team's job. A specialist may own the pipeline and the hardest tests, making sure the fastest checks run closest to the work and slower ones are sequenced after. But engineers, QA and agents all work against the same checks. Rubick even suggests a goal of shipping a small share of commits, starting with one percent, with no human verification at all, then expanding what's safe to allow-list.

How to start with verification engineering

1. Pick one kind of change. UI fixes, copy changes and small bugs are good first candidates.
2. Write down what "done" means. Make the success criteria specific enough that a check can confirm them.
3. Give agents a real environment. Each change needs an isolated sandbox with the app running, so checks test real behavior.
4. Add evidence to every change. Attach screenshots, recordings and test results to the pull request.
5. Keep a human on the merge. Route every agent change through your existing CI and required reviews.
6. Match checks to risk. Add heavier verification for payments, auth and data changes, and lighter checks for low-risk edits.

Then watch where reviewers still catch problems. Each catch is a check your system is missing.

How LightSprint fits

LightSprint builds verification into every change. Each change runs in an isolated cloud sandbox with a live preview, and agents produce verification artifacts such as screenshots and recordings so reviewers start from a change that already runs. The review hub brings the result, feedback and merge-readiness checks together.

Shared skills and guardrails give agents your team's conventions before they build. Builders, typically PMs, designers or ops, can propose changes and view previews but can't merge. Approvers, typically engineers or leads, review and merge through your existing GitHub rules and CI. Read more on our engineering page, or see What are cloud agents? for how the sandboxes work.

FAQ

What is verification engineering?

Verification engineering is the practice of building the checks, environments and evidence that prove AI-written code works before it ships. Engineers design deterministic feedback for coding agents and reviewers instead of reading every line by hand.

Why is verification engineering important now?

Coding agents produce code faster than people can review it. OpenAI reported a small team merging roughly 1,500 agent-written pull requests in five months. When writing code is no longer the slow part, proving it's correct becomes the bottleneck.

Is verification engineering the same as QA?

Not quite. Traditional QA often tests work after engineering hands it off. Verification engineering builds fast, automated feedback into the process itself, so agents and engineers check work as it's built, with specialists owning the pipeline and the hardest tests.

What is the difference between verification and validation?

Verification asks whether a change meets the agreed criteria. Validation asks whether those criteria solve the user's real problem. Verification can be automated with tests and evidence, but people stay responsible for validation.

Is verification engineering the same as hardware design verification?

No. In chip design, a design verification engineer tests hardware with tools like SystemVerilog and UVM. Verification engineering for software means building the checks that confirm AI coding agents' output is correct.

Start for free