Letting the Models Reason

I wrote in Sixty-Five Thousand Lines Out about copying the wiki engine into a clean folder and deleting unused code. That post stopped at the deletions. This one is about what I have been doing in that folder since: shrinking it further, and changing how the engine decides what the wiki should say.

I am still in that tree as I write this. The work is continuous tests, continuous bugs, and a deliberate move away from a deterministic style of building the wiki — one that AI models only supported — toward one the models lead.

01Why the rules kept losing

For a long time the engine was a thicket of heuristics. A Python function would cut a speech attribution off the end of a sentence, match name fragments, look at capitalisation, or decide that a shorter description was the same as a longer one because its words sat inside it. Models were asked for help, but the program still made the final call by comparing words and spans.

Every rule of that kind ran into an edge case. A book that put the speaker before the line. A pronoun that the grammar filter refused. Two rooms that shared a label. A family word that turned a person page into a group. Fixing one case grew another function; that function needed a guard; the guard needed an exception. The codebase became a sprawling monster, and I could not say with confidence which path had produced a given page.

The failure mode was not that the rules were badly written. It was that they were the wrong kind of thing. Prose does not obey a closed list of patterns. Models do what they do best when you let them reason over the passage, and they do what they do worst when you trap their answer behind a hand-written filter that discards it for the wrong reason.

02What the clean tree looks like now

The old tree holds 884 source files outside data, caches and build output, of which 534 are Python, for 184,217 lines of Python. That matches the roughly 184,000 figure I gave when I first counted the old tree after the September post.

The clean tree on the Desktop, texfaber-wiki-engine-clean, now holds 85 source files on the same basis, of which 31 are Python, for 26,618 lines of Python. The first copy on 22 October was 276 files and about 120,000 lines; the three deletion batches left about 57,000. It has kept getting smaller. There is a hard ceiling in the commit check: the root Python files may not grow past the size written in tools/engine-size (today 26,899), and new test files are seized rather than left in the tree.

03What the models decide

The standing rule in the clean tree is short enough to quote: models decide identity — who a printing is, who said a line, whether two pages are one, who takes part in an event. Python files a page, a line or a participant only by a model’s answer at exactly its own printing. It never joins, splits, skips or discards by comparing names, words or spans. What no model settled is an engine error, never a guess.

In practice that means a pipeline of paid steps after a free chapter split: readers and a decider; a check; dialogue; fold and hindsight; who-is-who across the book; continuity; event identity; event cast; then a free render. Haiku and Luna read. DeepSeek decides claims, descriptions, speakers, participants, filings, merges and contradictions. Jev confirms what he can. Settlers take what the checker and the others leave open. Anything still unsettled shows as unfinished work on the build, not as a silent default.

Recent commits have been deleting the old filters one by one. Who spoke a line is now the reader’s proposal and the judge’s decision, not a Python reading of attribution grammar — those helpers are gone. Whether quoted text is speech at all is decided the same way as any other claim; the grammar filter that sorted dialogue lines is gone. Who tells an event drawn from speech is proposed and judged, not inferred from the speaker whose line overlapped the quotation. A shorter description is no longer deleted because its words sit inside a longer one. A person page is no longer turned into a group because its last word was a plural family word. Three who-step rules that filed a page under a larger printing, skipped a printing as “the same phrase”, or lost a speaker over one unanswered nested printing are deleted.

Python’s job, in those places, is narrower: check that the mention exists, that the evidence is real and prints once, and that the answer lands on exactly that printing. That is filing, not reasoning.

04Testing, and the bugs it keeps finding

I do not keep a permanent test suite in this tree. Early on, about twenty-eight percent of it became tests; the commit check now seizes new ones. The check that matters is a whole novel nobody here has worked on, read against its pages, or a free rebuild from stored answers that must stay byte-identical when the change should not touch the wiki.

That still means constant testing. A two-chapter run of The Mysterious Affair at Styles on 4 November, and a build of The Man Who Was Thursday on the 5th, each threw up faults that the deterministic leftovers had been hiding: accepted event participations that never reached the wiki while the step reported complete; description spans left in window coordinates so half of them pointed at the wrong text; three identical “Yes,” lines collapsed into one; the fold showing Jev a passage chosen by name rather than the page’s own printing; a true description rejected by sentence distance; a speaker refused because another mention shared a name word. Most of those are fixed in the commits of 5 November. Some are not. The to-do list still names places where a Python rule decides identity, or where Jev settles alone without settlers, or where something can finish in silence.

I am not going to claim the wiki is more accurate yet. The clean tree was never going to buy that by itself, and neither does swapping a heuristic for a model call until a fresh novel is built and read. What I can say is that the failures are now the right shape: an unsettled question shows as an engine error, a discarded model answer is treated as a bug, and the next change is judged on a book that has not been tuned to.

05What this does not settle

A title page can still be read as a chapter if it is left above the first heading. Some steps still need a real novel before I will call the mocked proofs done.

The point of the shift is simpler than the machinery. I spent a long time trying to make the program out-reason the book with rules. The book always had another edge case. The new method is to stop competing with the models at the thing they are for, keep Python on the ground — quotes, addresses, ledgers, spend — and let the reasoning happen where it belongs.

Thanks for reading.

J.H.