Eighteen Continuity Checks. Fourteen of Them Never Call a Model.

Tim Shen · October 18, 2026 · 6 min read

The usual way to check a novel for continuity errors is to put as much of it as fits into a large context window and ask. It is a reasonable first move and it has three problems: it costs money per chapter, it takes seconds, and when it says something is wrong you cannot ask it which rule fired.

Guardian's consistency layer is eighteen named detectors instead. Fourteen of them never call a model.

The eighteen

Each one declares its own cost, the languages it is allowed to run in, and what triggers it. The local ones run on a debounce while you type; the model ones only on a deliberate deep run.

Local — no model call:

  • Dead character active — someone deceased is speaking or acting
  • Character name typo — a near-miss spelling of an established name
  • Name written inconsistently — the same person under two spellings
  • Character trait contradiction — a fixed attribute changed
  • Timeline conflict — events that cannot both be true in that order
  • Quote differs from source — a quotation that does not match what was quoted
  • Figure contradicts claim — the number and the sentence disagree
  • Knows something they shouldn't — information the character was never given
  • In two places at once — presence contradiction
  • Chapter numbers disagree — structure against itself
  • Chapter in a different person — first person in a third-person book
  • Tense shifts mid-chapter
  • Canon changed here — this passage silently moves established world state
  • Decided name missing from the translation — a term the translator fixed, dropped in the other column

Model call — only on a deep run:

  • World constraint violation — breaks a rule the world established
  • Contradicts a recorded fact
  • Entity field contradiction
  • Web sources disagree — for non-fiction, where the claim has an outside referent

The split is not about ambition. Most of that first group is decidable from the text plus a structured record of what the story has already established, and anything decidable that way should not cost a model call. What is left for the model is the part that needs reading comprehension.

Rules cry wolf, which is the actual problem

A continuity checker with poor precision is worse than none, because the writer learns to dismiss it without reading. Every note that is wrong taxes every note that is right.

We have two answers, and neither of them is "tune the prompt".

One: a detector declares where it does not work

Each detector carries a locale list. Six run everywhere. Ten run only on English and Chinese. Two — person shift and tense drift — run only on English.

That field exists because of what an audit found when the product gained Vietnamese. Two examples, both measured rather than predicted:

The dead-character rule tests whether a character is "acting alive" using an English verb list plus a Chinese one. Vietnamese matches neither, so a deceased character could say and do anything in a Vietnamese chapter and the rule would pass it, every time. The comment in the source calls that silence dressed as coverage, which is the right name for it: the check appears to run, reports nothing, and reporting nothing is indistinguishable from a clean chapter.

The name-typo rule was worse — not silent but wrong. Vietnamese given names and place syllables are often three-letter ASCII forms one edit apart, and the guard list protecting common words is English. It reported "Hai" as a typo of "Mai", "Han" of "Lan", and "Nam" — in "Nam Định" — of "Lam". Meanwhile it missed the actual Vietnamese typo class, a dropped tone mark, because it compares whole names.

Both are now declared off for Vietnamese. Not fixed — off, and visibly so, until someone writes the version that works. A detector that cannot work in your language should say so.

Two: a calibrated gate over what a rule emits

The harder case is a rule that works but over-fires. For those, the detector catalog can declare a gate: a small logistic model, plain JSON weights, pure TypeScript on the CPU, that scores each finding and drops the ones below a threshold.

Exactly one rule ships with a gate today. On held-out canon books it takes that rule's precision from 67% to 90.8% at 91.8% recall.

That is the number I would put on a slide, and it is the wrong one. The split is by book, and several authors appear on both sides of it. On an author-disjoint split the same gate reads 85.3% precision at 86.6% recall — still a large gain over 67%, and under our own 90% recall floor. Both numbers are in the source. The second one is the honest one.

Two more rules are trained and deliberately not gated, because they miss the bar on real prose. One of them taught us something general: the gate had learned tense, not syntax. The synthetic training pairs used present tense 57% of the time and the canon is almost entirely past tense, so "was a" became a keep signal. It scored beautifully on synthetic data and would have been useless on a novel.

No gate weights reach production except through a promotion script, and no model API is ever called to generate or label the training data — a test walks the transitive imports of every gate script to enforce that.

And a contract that the quiet case stays quiet

The failure mode nobody tests for is the clean document. A rule that fires on a page with nothing wrong with it destroys trust fastest, and it will never show up in a recall measurement.

We learned this from two notes on a real 1,879-word essay that did not come from the prose at all. They came from the step that turns the editor's HTML into text. An italic journal title before a comma — <em>Advances</em>, 10(28) — reached the punctuation rule as "Advances , 10(28)", because every inline tag became a space. And the paragraph-length rule read the entire body as one 1,805-word paragraph, because it split on blank lines while the editor separates paragraphs with a single newline.

Both had been in the pipeline for months. Nothing had ever run the rules over a document shaped the way the editor actually writes one.

So now something does. A contract test runs the local style bundle over a document carrying every mark the editor emits — italic, bold, link, entity mention, footnote superscript, headings, list items, an empty paragraph, hard breaks — around prose that is itself clean, plus a Chinese version with its own punctuation traps. The output must be empty. Not "mostly empty". Empty. A rule that fires there is reading markup, not writing.

Why this shape

Eighteen named detectors is more code than one prompt. What it buys:

A note can say which detector raised it, so a writer who disagrees can turn off that one rather than the feature. Fourteen of them cost nothing, so they can run while you type instead of when you ask. Each one can be measured on its own, which is how we know the Vietnamese problem was two different bugs and not one. And a detector that should not run in your situation can be declared off, out loud, instead of quietly passing everything.

The alternative — one model call that returns a list — can do none of those things, and its precision is whatever it is on the day.