Why your “percent AI-written” number is probably wrong

Ask three teams what share of their codebase an AI wrote and you will get three numbers produced by three different methods, none of them written down. The gap between those methods is not small. In real data we have seen a single file reported as sixty-five percent AI-written when the defensible figure was eight.

The four things people actually count

Lines a model emitted. Everything that ever appeared in a response. This is the biggest number and the least meaningful: it includes suggestions that were rejected, code that was rewritten immediately and examples that never left the chat.

Lines accepted into the working copy. Better, but it still counts code that was heavily edited before it shipped.

Lines a model produced that are still in the file. This is the number worth having. It measures AI authorship that survived review.

Commits touched by an AI session. A proxy that overstates dramatically — one suggested line makes an eight-hundred-line commit “AI-assisted”.

The mistake that inflates everything

The most common error is not distinguishing hand edits from model output. If your tooling records file changes, it records the ones a developer typed as well as the ones an agent produced. Counting both as AI activity is an easy mistake and a large one.

The fix is to treat them as separate sources. A manual edit explains why a line looks the way it does; it is not a prompt, and it does not belong in an AI contribution figure. Making that split is what moved that sixty-five percent file to eight.

What to do with the lines you cannot place

Every honest method leaves residue. Code written before you started recording cannot be attributed. Code that has been reformatted may no longer match what the model produced. A line that appears in several interactions has to be resolved by a rule rather than a fact.

Those lines belong in their own category. Folding them into the human column produces a tidier chart and a useless one — and the first person to test the number against a file they know will find it.

A definition worth publishing

Whatever you measure, write the definition next to the number. Ours reads roughly: a line is AI-authored if text produced by a model in a recorded interaction is still present in the file on disk; lines changed by hand afterwards are human-authored; lines that cannot be matched are unknown; unknown is reported separately and never redistributed.

That definition is arguable, which is the point. A figure nobody can argue with is usually a figure nobody checked.

Measure it properly on one repository.

The gap between the raw number and the defensible one is usually the interesting part.