Is Increasing AI Usage Increasing Code Review Burden?

AI usage at Trellis

It has been six months since we adopted Claude Code at Trellis. Some of us adopted it quicker than others, but we are all consistently using it now.

There are a lot of opinions out there about how good or bad the output is and how much faster (or slower) it makes a developer. I remember, in recent history (before the advent of these coding harnesses), the idea of a 10x developer becoming a meme, more because of the stuff surrounding it than anything else.

Now that is coming back with a vengeance, except now we are talking about 100x and 1000x developers because of these AI coding tools. I personally find it hard to find real-world examples of how teams are using it, how it is affecting their productivity, output, quality, LOC, or any other metric you can "measure" a development team by. I put measures in quotes because I don't particularly subscribe to the idea that individual quantitative metrics are a good way to measure developer productivity.

We are a small team (four developers total), and each of us generally is working on very different tasks in terms of complexity, size and area/level of the platform. In the analysis below I used PR count, lines of code, issue count, and token usage as crude approximations to ensure that no individual was being crushed by the AI usage of other developers.

At Trellis, I try to focus more on team, product and company level metrics (grounded in DORA) to understand developer productivity, output and/or quality:

1. Is the platform stable? Low to zero Change Failure Rate, no outages and/or incidents, etc.
2. Hitting our product roadmap targets we set out.
3. Lead Time for Change – how long does it take us to deploy a change to production from when it was merged.
4. Deployment Frequency – how often do we deploy changes to production?

With all of that being said, let's get into it.

Is our increasing AI usage increasing the PR review burden of human developers?

This is the main question I wanted to ask. We have a strong culture of code reviews at Trellis, and while we have a nearly fully automated deployment process centered around our DORA metrics that has forced us to build out a robust testing strategy, we still ensure that (almost) all code is reviewed before it gets merged.

Note: I said almost because there are still some that do not get reviewed. They are not product runtime code though; they are generally tooling, DevOps updates, package updates, and documentation. Low-stakes changes like that.

Beyond that, there were three sub-questions I wanted answered to provide a more detailed picture of how AI usage was affecting our team and process.

1. Does AI-generated code draw more requested changes per PR?
2. Does heavy AI use by some drag down anyone else?
3. How much of human review feedback could tools enforce instead?

Note: Throughout this analysis one "requested change" is equal to one review thread opened by a human reviewer other than the PR author.

The data is split into two buckets: Jun–Nov 2025 and Dec 2025 onward, referred to as pre-AI and during-AI.

High Level Results:

Overall, we increased our output by 55% while review friction stayed flat. Big win!

That being said, it's not quite as rosy as it sounds. Yes, friction per reviewed PR stayed flat, but there were more PRs, which does increase the amount of reviewing from each team member. This is where the "addressable by automation" metric comes in, i.e. Question #3: How much of human review feedback could tools enforce instead?

At an absolute minimum, 11% of human review comments could be enforced by tools like linters, formatters or other deterministic systems.

Note: All July data is partial up to the writing of this post.

Output went up. Review friction didn't.

Bars: PRs merged per month (left axis). Line: human-requested changes per PR (right axis). If AI were taxing reviewers, the line would climb with the bars.

As mentioned previously, December is where we started using AI coding harnesses heavily. It is also our slowest month of the year for development (many of us take 1–3 weeks off around Christmas). This gives us a nice before-and-after.

As you can see, per month the number of PRs we are merging grows while the human-requested changes per PR stay flat and start to trend down a bit (especially in June). Absolute review comments grew by about 19%, whereas output grew 55%. Per unit of code shipped, the human review burden fell.

Accounting for non-reviewed PRs

The all-PR average falls, but the reviewed-only average is statistically flat.

To ensure that the increase in self-merges (generally from me building tooling, lint rules, documentation updates, etc.) didn't skew our results, this analysis considers only PRs that were reviewed by a human to not drag all of our averages down and make the metrics look a lot better than they actually are.

Whereas all PRs saw a drop from 0.67 → 0.51 average comments per PR, reviewed PRs only saw a drop from 0.89 → 0.84, which isn't statistically significant enough to make any kind of determination. But it is a win in itself, as it shows that AI isn't producing code that requires more changes than entirely human-written code.

Sub-question 1: does AI code attract more review comments?

Each dot is one developer-month (Feb–May 2026, the window with per-developer attribution data).

This one was a little harder to analyze as I was not able to pull significant amounts of data, so there is a small n and it should be taken with a grain of salt. Maybe one day Anthropic will give us a way to query for all token usage data so I can do a full retroactive analysis across the entire team.

What we can see here is that there is no significant relationship in either direction that would allow us to draw a causal conclusion (r −0.14 to +0.15).

While there isn't a lot of data to go off of, we can see that the heaviest AI months are at or below the average friction per PR.

Sub-question 2: did anyone drown in everyone else's AI output?

Team-level monthly averages, on a shared scale. The volume of code shipped each month grew almost three times faster than the human review workload it created.

Our fear with increasing AI adoption is that a spike in output from one or more team members would drag their teammates down because of the explosion in output and the need for more reviews.

While I won't share developer-specific metrics, every developer's output is within monthly noise. We need more data to make hard determinations here right now, but with the limited information we have, it does not seem like any individual is being dragged down by PR reviews.

Some high-level anonymized metrics:

1. Almost everyone's changes received per PR went down.
2. Almost everyone's Lines of code per PR went up.
3. Half the team's PRs/mo went up, the other half went down slightly (in the single digits of PRs), but that half generally works on more complex/investigatory work weekly.
4. Automation increased its share of all PR review comments. Up to 50% from 31%.

One thing we will be keeping an eye on is that we have one team member who is responsible for most of the frontend work and is usually the primary reviewer for the frontend PRs. We need to be cognizant that, with the ability to increase our output, we do not overwhelm them as others take on full-stack work.

Where the review comments went instead

Automated reviewers (specifically our in-house AI review bot) now write about half of all review comments, up from 31% pre-AI.

Pre-December we had been using Codegen.com for a bit of agentic coding, but mostly for code reviews. Once Codegen got acquired by ClickUp we lost access to it in December and moved our automated code reviews mostly to Sentry Seer. We used that until the trial ran out, we liked it but wanted something a little more tailored to us and controlled in our CI without needing to pay a per-seat cost.

We looked into a few other options, but I decided to ultimately build our own using Command Code and open-weight models.

Note: Between some large vacations and the fact that we stopped using Sentry Seer, it is a bit of an outlier month.

After implementing our own AI code reviewer, we saw the share of review comments being made by developers vs. bots start to drop. Now, the AI code review bots are accounting for the majority of review comments.

It wasn't just as easy as us reviewing less and relying on the AI code reviewer more. It took a lot of tweaking. What can the AI code reviewer successfully flag? What can be flagged via deterministic tools? What can be flagged via custom lint rules? What kinds of issues need to be flagged by a human?

This took many iterations, but thanks to AI, I was able to build out an extensive system of rules:

1. Aggressively started adopting correctness lint rules, moving as many over to Oxlint as possible, and enforcing oxlint to be run pre-commit (it is so fast that it doesn't affect the DX of committing),
2. Started building out custom lint rules using the Oxlint JS Plugin API, for patterns, naming schemes, and other Trellis specific requirements.
3. Creating and tuning the Taste file for Command Code so the harness responsible for running the open-weight models understands our coding patterns.
4. Building out ADRs (Architecture Decision Records) and SOPs (Standard Operating Procedures) to flag when we are going against an agreed-upon architecture or breaking a process (for example, feature flag removals).
5. Massively overhauling all existing documentation and structuring it in a way that the agents can leverage progressive disclosure.

Sub-question 3: how much review can tooling actually absorb?

Every actionable human review comment from 13 months, attributed to the cheapest mechanism that could have caught it, with the four numbers that matter: the automation ceiling, the deterministic floor, the AI band, and the irreducible human core.

One of my main goals was to make it so that the human reviewers didn't need to worry about reviewing for nits. I (and they) would rather spend their time on their own work rather than having to point out little things.

Of the ~1300 review change requests made, about 66% of them could be covered by some level of automation. This includes:

1. Linting (Custom or built-in across oxlint and eslint)
2. Formatting
3. CI scripts and other automated checks
4. AI code review

As it currently stands, deterministic tools can enforce about 11% of the analyzed comments, as a floor. We are actively working on trying to cover more of these using linting tools and building custom lint rules where one does not exist. It is now incredibly easy to have these coding agents build out Oxlint JS Plugins for specific use cases in our codebase thus allowing us to enforce more of these rules before they ever get pushed up and/or have a review requested.

About 55% of the analyzed PR review comments could be addressed by the AI code review. But this is a ceiling as that system in itself is not deterministic. Much like humans, it is still possible to "miss" things.

The other 34% are where, for at least the time being, I still want a human: novel design, intent, domain context, deploy coordination.

The story of our automation stack

None of this happened in one sitting. It took many iterations, tools, and attempts to get where we are now (and we still have room to improve): you have to build the floor before you build anything fancier on top of it.

The first stuff we shipped had nothing to do with AI at all. We did a bunch of work to ensure we could flip on as many TypeScript strict flags as possible and pinned our dependencies to exact versions instead of ranges, moved to an isolated node_modules linking strategy with Pnpm, and added hooks that fail if generated GraphQL/Prisma code/migrations drifted. It is all not very exciting to talk about, but it is why we are able to trust what we built on top of it later.

Once that floor was solid, we started pushing correctness rules as deep into deterministic tooling as they'd go. Instead of turning on individual oxlint rules one at a time, we enabled whole categories at once (this is still in progress as we enable the default recommended rules in oxlint). We standardized on one style of null/equality checking, so it stopped being a matter of personal taste, normalized our commenting conventions, enforced consistent code spacing, alphabetized our GraphQL files, and banned known-abandoned or unmaintained packages outright. None of these need an AI or LLM anywhere near them, but they also set up the standards and patterns that the models will follow.

That's about as far as an off-the-shelf linter goes (whether that be oxlint or eslint), though. Anything specific to how we actually build at Trellis had to be written by us. So the next step was custom lint rules. Using the Oxlint JS Plugin API to encode "how we do things here" instead of generic best practices:

1. Rules enforcing our GraphQL file/library structure and which library a given piece of code is allowed to live in
2. Rules that catch untyped or unsafe test assertions, and (still more tuning ahead of it) discouraging thin, low-value spec files.

These are similar to the above: fully deterministic, zero trust required, just repo-specific instead of universal.

Everything up to this point is stuff a linter can enforce deterministically. The AI reviewer is the opposite end of that spectrum. The AI reviewer runs on every PR in CI and leaves inline suggestions, and we deliberately kept it non-blocking while we earned trust in it.

It can comment and approve. However, it cannot "Request Changes", so as not to block PRs from being merged while we tested it out and made sure it was providing value and not just slowing us down. (All PRs still require human reviewers, so the risk was low.)

We didn't start from a blank slate, though: we mined roughly 2,200 of our own past PR review comments to build a taste profile, so it flags things the way we (the human reviewers) actually flag them instead of some generic checklist.

Additionally, I went back through our own recurring feedback and was able to surface 116 patterns that kept showing up often enough to be worth acting on systematically. For each one we had to make a call: is this exact enough to become a deterministic lint rule, or does it need the kind of judgment only the AI reviewer (or a human) can bring?

That split is basically the whole point of this post: linters are the floor, exact and repeatable; the AI reviewer is the ceiling, broad but probabilistic; and humans are still the layer above both. Our goal is to shift the review burden as close to the floor as possible.

Watching the AI reviewer find its level

Comments per merged PR, daily, with a 7-day rolling average. Each dashed line is a shipped tuning change.

Initially during launch week it was running hot at about 1.9/PR. Lots of this was noise, or was unneeded because a linting rule already covered the same case, among other problems like flagging things that were not incorrect.

The taste profile was initially trained on all the PR review conversations and requested changes over the past year-ish. This required some additional tuning though:

1. Used a system prompt to be more specific about what documentation to follow.
2. Continually tuned the taste profile based on all the AI reviews that were coming in, so we were catching AI flags that were not quite right as fast as possible.
3. Added a max files parameter to skip reviews on large PRs as a fail-safe as sometimes we do a repo wide update when putting in a new lint rule.

Through these tweaks we about halved the volume of PR comments from the AI code reviewer. We did not lose any value, though, as the comments that got reduced were the ones that were unnecessary anyway.

We are now sitting at about 1.35/PR, ~42% of PRs get at least one comment.

So has AI been a net positive for us?

I can confidently say that I think it has been. Has there been some friction along the way as we get guardrails in place to ensure we don't slide backwards on the quality we have spent so long putting in place and building within the team? Absolutely, but we are making more and more progress every day, and these automated systems we are building just keep getting better, which allows us to focus on providing value to our customers.

Overall, we found that:

1. Friction per reviewed PR remained flat.
2. No positive AI ↔ Friction correlation.
3. No individuals declined in their own output.

All while the team's output rose significantly.

As a reminder, this is all correlation right now as AI adoption, tooling rollouts, varying size and complexity of work, and projects being started and stopped are all happening simultaneously.

Where are we going next?

The main area we will be focusing on is lifting the deterministic floor. If we can enforce more and more of our patterns and conventions in a way where the developers trust that they do not need to keep brain space for those rules when doing code reviews, that speeds them up and frees up brain power for other tasks.