Why AI detector scores are evidence, not a verdict

A detector estimates similarity, not provenance

Most general-purpose detectors classify linguistic patterns learned from examples of human and model-written text. A high score means the text resembles patterns in the detector's training and evaluation data. It does not identify the author, tool, prompt, or editing history.

A genuinely verified watermark is a different mechanism. It depends on a specific generation-time design and corresponding detector—not a generic ‘AI percentage.’

False positives are not an edge case

OpenAI retired its own text classifier after publishing weak detection performance and warning that the tool should not be used as a primary decision-making mechanism. Turnitin similarly tells reviewers that its score is not definitive evidence and suppresses precise low-range scores because of false-positive risk.

Formulaic professional prose, edited text, non-native English writing, short passages, and text from an unseen model can all shift results. A score without domain-specific calibration is easy to overread.

Evasion and rewriting expose a second weakness

The RAID benchmark found that many detectors generalize poorly to unseen models and domains and can be weakened by adversarial changes. This means a low score does not prove human authorship either.

Optimizing a rewrite against one detector creates a circular product claim: the same system rewrites the text and declares its own output successful. A better review target is semantic fidelity, evidence, readability, and voice match.

Use a responsible review protocol

For low-stakes editorial review, a detector can point to passages worth inspecting. Show sentence-level signals, the basis for the score, qualifying word count, language, method version, and a clear uncertainty label. Compare with writing history, drafts, citations, and process evidence.

For education, employment, publishing, or discipline, require human review and corroborating evidence. Give the writer a fair way to explain their process and challenge the result.

• Never report ‘X% AI’ as a literal percentage of authorship.
• Do not compare scores across detectors as though they share a scale.
• Keep model version, language, passage length, and date with every result.
• Do not use detection as the sole basis for a consequential decision.