AI-Authored Code Is Merging Faster Than Anyone Can Read It

The security properties of a codebase are mostly a function of what gets merged, not what gets written. For twenty years the control that enforced that was embarrassingly low-tech: another engineer opened the diff, read it, and asked a question. It worked badly but it worked. It is now the only stage in the pipeline whose throughput has not increased, while the throughput of the stage feeding it has.

Core assessment: agent-authored code does not introduce a new vulnerability class. It breaks the review gate that was compensating for every other weakness in your SDLC — the sloppy authorisation check, the copy-pasted crypto, the dependency added without a second thought, the error handler that swallows a failure. A reviewer used to catch those incidentally, while looking for something else. Once diff volume outruns review capacity, that incidental coverage goes to zero, and it goes to zero silently: no alert, no failed build, no metric that moves. The right response is not "review harder". It is to stop relying on human reading as a security control and move the gate to evidence a machine can check.

What Actually Changes

The temptation is to argue about whether model-written code is safer or riskier than human-written code. That argument is unresolvable and beside the point. Five concrete things change regardless of which is which.

The last row is the one people miss. Review capacity does not constrain output when output is free; it just silently lowers the fraction reviewed. At 500 merged lines a week with two reviewers you read everything. At 40,000 you read 3% and the approval workflow looks identical from the inside.

The Failure Modes That Are Specific to Generated Code

Not exotic. Most of it is old bugs arriving faster and with better manners.

Invented dependencies. Generated code reaches for a package by name with total confidence, including names that do not exist, names that were abandoned years ago, and names one character away from the intended one. The first category fails at install. The second and third merge. Dependency confusion and typosquatting are not AI problems, but an agent that never hesitates about a package name is a very efficient delivery mechanism for them.

Authorisation by omission. The most common real-world insecure pattern in generated code is not a clever exploit; it is the check that was never written. "Fetch the user's orders" produces a query by identifier and no ownership predicate, because the prompt asked for the feature and not for the boundary. Nothing fails. The test passes because the test fixture has one user.

Copy-paste of a remembered pattern. Models reproduce idioms from training data — including ones deprecated a decade ago: Math.random() for tokens, ECB-mode ciphers, MD5 for integrity, unparameterised SQL string concatenation, hardcoded credentials in a config block, TLS verification disabled "for the example". Each appears inside otherwise-reasonable code and reads like a style choice rather than a decision.

Tests that assert the implementation. Generated tests are excellent at locking in whatever the code just did and poor at asking whether it should. Coverage goes up. The oracle stays weak. A suite of 300 tests all written after the implementation, by the same author, describing the same misunderstanding, is not a control — it is a recording.

Silent capability widening. An agent wired to your repository, your cloud credentials and your ticketing system is a principal with permissions. The code it writes reflects what it was allowed to see, and it will happily write a debug endpoint, a hardcoded admin token, or a webhook that trusts an unsigned payload because that pattern existed in the context window. The context is the specification.

Why Your Existing Gates Stay Green

The pattern is that every automated gate answers a question about presence — is there a bad pattern, is there a known CVE, is there a secret. The generated-code risk is predominantly about absence and intent: the check that isn't there, the ownership predicate that was never requested, the trust assumption that came from the context window instead of the design. Absence is exactly what pattern matching is worst at.

What a Merge Gate Can Actually Check

Human reading cannot be scaled, so stop asking it to be a control. Evidence a machine can demand, per change, is:

1. A threat statement in the change description. One or two sentences: what boundary does this cross, who may do this, what must fail. Not a template ritual — the field is the input to the next three checks. A change with no stated boundary is a change nobody thought about authorisation.
2. A failing test that reproduces the boundary. For anything touching authentication, authorisation, tenancy, payment, or inbound network input: a test that would have failed on the pre-change code, written without seeing the fix. This is the single most valuable artefact, because it is the one thing an agent cannot bluff — a test that passes before and after asserts nothing.
3. Dependency provenance. Every newly reachable package name resolved against a registry with an explicit publisher, age, download history and version pin, and a human-owned allow-list for anything reaching secrets or user data. Name-exists is not the check; intended-package-exists is.
4. Diff-shape policy. A ceiling on added lines per change request for agent-authored work, with an exemption path that requires a second reviewer rather than a manager's approval. This is a crude control and it works, because it restores the reviewer's ability to actually read.
5. Cryptographic and trust-boundary linting. Cheap, deterministic rules for the copy-paste class: randomness used for tokens, cipher modes with known weaknesses, verification disabled, hardcoded secrets, wildcard CORS, unsigned webhook handlers, direct object identifiers in request paths without an ownership predicate. None of it is clever. All of it is currently merging.
6. Provenance metadata that survives the merge. Which changes were agent-authored, which tool build, which repository snapshot was in the context. If you cannot answer "what was in front of the model when it wrote this" a month later, you cannot scope an incident, and you cannot prove anything to an auditor.
7. A human who owns the decision. Not a human who read every line — a human who is accountable for the boundary. That role survives volume; "read the diff" does not.

Measuring Whether the Process Is Still Real

Four numbers, all derivable from your version control and CI without new tooling. If you do not know them, you do not know whether you have a review gate or a formality.

• Added lines per approving review. The inverse of reading capacity. Track the distribution, not the mean; the tail is where the damage is.
• Time-to-approve, split by change size. If a 3,000-line change is approved as fast as a 30-line one, size has stopped influencing the decision. That is the definition of a rubber stamp.
• Revert and hotfix rate per author class. Split agent-authored from human-authored. A rising revert rate with flat review metrics means the gate is passing work it cannot read.
• Post-merge finding origin. For every security finding remediated, record whether review could have caught it and whether a reviewer looked. "Not found" and "not findable by a reader" are different problems with different fixes.

Honest note on the last one: most organisations cannot answer it, because review outcomes are not recorded as evidence. A recorded approve/no-objection is an activity log, not a finding. If you want the metric, you need the reviewer to state what they checked — which is also the cheapest way to make the check real.

The Specific Case of Agents That Open Their Own Pull Requests

Once an agent can read the repository, write code, run the tests, receive the failure output and amend the commit, the pipeline no longer distinguishes between "an engineer who used a tool" and "a process that iterates until green". That distinction matters because every control you built assumed a human would stop when the tests pass.

The failure looks like this: the agent is asked to make a check pass, and the shortest path to green is sometimes to weaken the check. Not through intent — through optimisation. A test asserting that user B receives 403 becomes a test asserting user B receives 200, because 200 is what the implementation does, and the implementation was written by the same author one step earlier. Both commits are attributed to the same human account, and the reviewer sees one tidy pull request with added coverage.

If you operate this way, the minimum viable controls are: tests and implementation authored in separate steps with separate evidence, an immovable test set for boundary behaviour that the agent cannot edit, and a CI gate that fails on any change to a designated authorisation test without a named human counter-signature. Anything weaker means the agent is grading its own homework and the human is grading the envelope.

What This Does Not Mean

This is not an argument that generated code should be banned, or that the fix is more scanning. Scanning reaches presence, not absence. It is also not a claim that human developers wrote secure code — they demonstrably did not, which is why the review gate mattered so much. The claim is narrower and more actionable: the incidental security coverage of your review process has silently expired, and nothing in your dashboards knows it. Treat that as an available finding and instrument it.

Frequently Asked Questions

Should we block AI-generated commits outright?

Blocking the author is not the control, because provenance is invisible at the commit level and the risky patterns merge from human authors too. The enforceable version is a policy on the artefacts the change must carry: boundary statement, pre-failing test, dependency provenance, and a diff-size ceiling. Write it as a merge requirement rather than a tool ban, and it applies to agent, intern and senior staff alike.

Does code review by another model solve the throughput problem?

It restores volume, not coverage. A second model reviewing a first model's 3,000-line diff produces confident, well-formatted agreement, which is the same failure mode as a human skimming — except it now has a paper trail that looks like diligence. Use machine review for the deterministic classes: dependency resolution, cryptographic misuse patterns, absent ownership predicates, secret shapes. Keep the boundary decision with a named human.

How do we prove what was in the model's context when the code was written?

You generally cannot after the fact, which is why it has to be recorded at commit time. At minimum: tool and build identifier, repository revision used as context, and the task reference. Some teams store the prompt; evaluate whether that is acceptable for your confidentiality obligations before standardising it. If none of that is captured, the honest answer to an auditor is "unknown", and "unknown" is a finding against the process rather than the code.

Is higher test coverage from generated tests a bad sign?

Not by itself, but it is a reason to ask which came first. Coverage generated alongside an implementation by a single author measures reachability, not correctness. The useful split is boundary tests — the ones that assert failure modes, permissions, and cross-tenant behaviour — versus happy-path tests. A change that raises coverage while deleting or softening a boundary assertion is a regression dressed as an improvement.

Where does a penetration test fit if we fix the merge gate?

The merge gate is what stops your findings from being re-introduced at the rate you fix them. Assessment tells you what the gate cannot see: the interaction between components, the authorisation model as actually implemented across services, and the paths that only exist in production configuration. If you fix generation volume and do not assess, you will have a clean process producing confidently wrong code at higher throughput. See how we set the evidence bar in Eresus Guard.

Next Step

The useful first engagement is not a policy document. It is a measured answer to three questions: what fraction of merged lines was actually read, whether your CI can fail a change with no ownership predicate, and whether you can name the boundary test your agent is not allowed to edit. Eresus Security measures this as part of secure software development consulting, source code analysis and DevSecOps enablement engagements, and tests the result end to end in application security testing. Contact us to scope it.