How AI Detection Tools Actually Work

Paste the same paragraph into three different AI detectors and you'll often get three different answers: "90% AI," "likely human," "mixed." That isn't a bug in one of them — it's what happens when several fundamentally different measurement techniques all get compressed into a single confident-looking percentage.

Here's what's actually going on inside these tools, and how to read their output sensibly.

Technique 1: Predictability scoring (perplexity)

The oldest approach asks a simple question: how surprising is each word, given the words before it?

Language models work by predicting the next token, so their own output tends to sit in the high-probability regions of language — common words in common arrangements. Human writing is messier: we make unusual word choices, coin phrases, and break patterns. A detector can run the text through a reference model and average the surprise (this quantity is called perplexity). Low perplexity — the text was highly predictable — correlates with machine generation.

A related measure is burstiness: how much sentence length and complexity vary across the text. Humans write in bursts — a long winding sentence, then a short one. Like this. Models tend toward unnervingly even rhythms unless pushed.

These signals are real, but they're weak. A careful human writer producing clear, plain prose can score as "predictable." A model asked to write erratically can score as "bursty." Perplexity measures a style, not an origin.

Technique 2: Trained classifiers

Most commercial detectors add a second layer: a machine-learning classifier trained on labeled examples of human and AI text. Instead of a hand-picked signal like perplexity, the classifier learns its own features — hundreds of subtle distributional cues no one designed explicitly.

Classifiers are stronger than raw perplexity scoring, but they inherit the biases of their training data. Research has repeatedly found two failure modes:

• False positives on non-native English writing. Writers using simpler, more conventional English phrasing — common among non-native speakers — produce text that looks like the "AI" side of the training distribution.
• Decay against newer models. A classifier trained on last year's model outputs is quietly measuring "does this resemble last year's models," which is not the same question as "was this machine-written."

Technique 3: Watermark verification

The third approach is categorically different — and far more reliable where it applies. If the text was produced by a model that embeds a statistical watermark, the provider can check for that watermark using their secret key. This is a direct measurement of origin rather than an inference from style.

The catch: only the provider holds the key, watermarks are only present if the provider enabled them, and the watermark degrades when the text is edited or rewritten. So even this strongest technique covers only a slice of real-world text.

Why detectors disagree

Now the opening puzzle resolves itself. Different detectors use different mixes of these techniques, trained on different data, calibrated to different trade-offs between false positives and false negatives. A conservative detector that hates accusing humans will miss more AI text; an aggressive one flags more AI text and more innocent humans. There's no shared scale — "87% AI" means "87% on this vendor's private scale, against this vendor's training data." It's also why a text can "pass" one tool and "fail" another without anyone lying: each tool answers a slightly different question, then presents the answer in the same confident percentage format.

Short texts make everything worse. Under roughly 300 words there simply isn't enough statistical material for any of these methods to work reliably, and vendors' own documentation usually admits as much in the fine print.

What a detector score actually tells you

A reasonable way to read a detector result: it's a weak hint about stylistic predictability, not evidence of authorship. It can justify a conversation ("this draft reads as very generic — did you write it yourself?") but not an accusation, and treating it as proof of cheating has caused real harm to students and writers whose only offense was a tidy prose style.

If you're on the other side of the equation — you used AI as a drafting tool and the output reads as flat and generic — the productive response isn't chasing a detector score. It's improving the text: vary the rhythm, add specifics only you know, restructure sentences so they stop following the model's default cadence. Those edits make the text better and less predictable at the same time. Our text rewriting tool automates the structural part of that — multiple passes over vocabulary and sentence structure, with an honest report of how much changed — but the specifics and judgment still come from you.

The bottom line

AI detectors measure statistical style signatures and, occasionally, cryptographic watermarks. They're useful as rough signals and unreliable as verdicts — especially on short texts, non-native writing, and heavily edited drafts. Read the score, note the uncertainty, and never confuse "predictable prose" with "machine-made."