Blog
One plain-language post per paper: the problem, what we did, what we found, and what is still open.
Long LLM outputs hinge on a few decisions, not on their length
About 9% of tokens carry long-range dependency and errors cluster, so the predicted decay of long LLM outputs is far gentler than (1 − e)^n.
No finite fix list covers every LLM failure, and a deployment doesn't need one
Open-ended LLM use has no finite fix list, but inside one deployment failures tend to recur, so a slowly growing library of fixes covers the most common ones.
Production AI learns outside the weights, without the optimizer that layer needs
Teams fix LLM systems by editing prompts, memory, skills and tools. Across ~130 systems, that loop runs mostly as patchwork, its cross-org optimizer missing.
Rewriting a prompt loses fewer details than deleting its tokens
Telegraph English rewrites text one claim per line with explicit symbols. Against LLMLingua-2 it lost fewer answers, most visibly on fine-detail questions.
Rewriting a retrieved passage beats cutting it to the same token budget
At a fixed token budget, rewriting passages as entity-relation clauses beat truncated and thinned text on three multi-hop datasets, and a summary on one.
A compressed code can pass its own checker and still change the facts
Learned symbolic codes passed their own checker while changing what the source said. Scored against the source, a compact format kept more per byte.
Asked for a quarter of the context, one encoder sent back two thirds
We followed Telegraph English rewrites from token budget to reader: 33 of 9,600 landed in band, and a compiler that held its ceiling still scored poorly.
Finding most of what an edit changed is not finding all of it
We scored whether rankings recover every statement an edit changes, not just most. Recall and completion disagree, sometimes at the edited sentence itself.
A linter that redoes the work catches injected errors that self-verification misses
Written in seven tags, a model's math reasoning can be checked by a SymPy-backed linter. It caught 195 of 196 injected errors; self-verification, 66.3%.
A math trace that passes our checker is right about one time in three
We checked a small model's math traces with a program. Accepted traces were right about one time in three; switching to the trace format cost accuracy.
In GRPO, the weight on an additive process reward acts like a switch, not a dial
Group standardization cancels an additive process reward's weight when outcomes agree and swamps the term when they differ. We measured both, then tested fixes.
An LLM labeler scores higher when its sibling wrote the answer key
Hold an LLM labeler fixed and swap who wrote its answer key: agreement rises when the key comes from a sibling model, and no human gold is needed to see it.
Most calls never reach the LLM, and the ones that do need a different prompt
A fine-tuned encoder answers roughly 87% of customer calls. The rest go to an LLM with five candidate labels, and that shortlist needs a prompt of its own.
Rare call issues can get lost before an LLM ever sees them
A 1% call issue can be absent from the sample an LLM proposes topics from. Over-splitting each category found 234 subcategories; flat clustering found 111.
Clarify a call once, then choose what each reader sees
We rewrite each customer call once into short, tagged statements. An encoder gains from clearer text alone; weaker prompted readers also gain from selection.