Independent researcher
How errors accumulate in LLM systems, and how much of a prompt a model needs.
Mikhail L Arbuzov works on two questions that turned out to be connected: how errors build up as a language model generates, and how much of a prompt the model actually needs to read. The first produced the error-accumulation trilogy, which moves from a critique of the exponential-decay picture, through a theory of where errors cluster, to how production systems learn outside the model weights. The second produced Telegraph English, a structured symbolic rewrite of prompts.
Mikhail is first author on Beyond Exponential Decay, The Architecture of Errors, Frontier and Localhost, Telegraph English, and three applied papers on conversation analytics at scale (The Same-Family Halo, High-Load Budgeted Categorization, and Clarify, Then Focus). Mikhail is also a co-author on the compression and reasoning papers and on Clusters Are Proposals.
to fill: Background, current focus, photo, personal links.Asked to rewrite passages into relational text at a set token budget, two LLM encoders landed in the registered band on 33 of 9,600 outputs, and their median output ran long. Compressors should be compared at delivered lengths.
An LLM labeler agrees more with silver labels written by a sibling model than with labels from other families. The contrast needs no human gold, so same-family consensus can be audited, and discounted, where gold is scarce.
A confidence gate lets an encoder answer roughly 87% of customer calls; with a five-label shortlist for the rest, cost falls more than 90% below an LLM reading the full list on every call. The shortlist needs its own prompt.
Recursively over-splitting each call category, then merging only the pairs an LLM confirms are duplicates, surfaced 234 subcategories against 111 from flat clustering. Redundancy was cheap, 10 merges among 244 proposals; coverage was not.
Rewriting each call once into short, attributed, tagged statements improved a supervised encoder without selection, and tag-based selection helped several weaker prompted readers further. With a distilled 0.6B normalizer, no large model sits in the serving path.
A symbolic code's own checker accepted translations that changed the source. Scored against a constructed reference, trace translators kept less per byte than a compact structured format, and larger translators left the gap open.
Asking whether a ranking finds every statement an edit changes, not just most, reorders systems. Embedding reranking nudged BM25's recall up and completed fewer sets; one model scored well on recall while missing the edited sentence itself.
On a small model's math reasoning, traces a deterministic checker accepted had the right answer about one time in three, and training the model to write checkable traces raised acceptance while lowering accuracy.
No finite fix list covers every failure mode of open-ended LLM use. Inside one deployment, published taxonomies suggest failures recur in a small catalogue, so a sufficient fix library grows slowly with sequence length, then levels off.
Under GRPO's per-group standardization, a process term added to the outcome reward cannot be tuned: its weight cancels where a group's outcomes agree and is swamped where they differ. Endpoint gains need replication and a shuffle control.
Production LLM systems are increasingly fixed by editing prompts, rules, memories, skills and tools, not weights. Across about 130 surveyed systems, that loop runs mostly as patchwork, and none governs its fixes across organisations.
Rewriting retrieved passages as pipe-separated entity-relation clauses, entities kept verbatim, beat character deletion, truncation and random subsampling at the same token budget on three multi-hop benchmarks. At a fixed budget, the form of the kept text is not a detail.
If a model writes its math reasoning as tagged lines, a rule-based linter can recompute equations and check claims in SymPy instead of asking a model to reread them. It caught 195 of 196 injected errors; self-verification, about two thirds.
Treat every token as an equal, independent chance to fail and long outputs look doomed. Published measurements find about 9% of tokens depend on long-range context, and errors correlate, so reliability tracks key decisions, not output length.
Rewriting text so each line holds one claim, with explicit symbols for cause and contrast, cut tokens by about two fifths and lost fewer answers than LLMLingua-2's token deletion, by the widest margin on fine details.
About 9% of tokens carry long-range dependency and errors cluster, so the predicted decay of long LLM outputs is far gentler than (1 − e)^n.
Open-ended LLM use has no finite fix list, but inside one deployment failures tend to recur, so a slowly growing library of fixes covers the most common ones.
Teams fix LLM systems by editing prompts, memory, skills and tools. Across ~130 systems, that loop runs mostly as patchwork, its cross-org optimizer missing.
Telegraph English rewrites text one claim per line with explicit symbols. Against LLMLingua-2 it lost fewer answers, most visibly on fine-detail questions.
At a fixed token budget, rewriting passages as entity-relation clauses beat truncated and thinned text on three multi-hop datasets, and a summary on one.
Learned symbolic codes passed their own checker while changing what the source said. Scored against the source, a compact format kept more per byte.
We followed Telegraph English rewrites from token budget to reader: 33 of 9,600 landed in band, and a compiler that held its ceiling still scored poorly.
We scored whether rankings recover every statement an edit changes, not just most. Recall and completion disagree, sometimes at the edited sentence itself.
Written in seven tags, a model's math reasoning can be checked by a SymPy-backed linter. It caught 195 of 196 injected errors; self-verification, 66.3%.
We checked a small model's math traces with a program. Accepted traces were right about one time in three; switching to the trace format cost accuracy.
Group standardization cancels an additive process reward's weight when outcomes agree and swamps the term when they differ. We measured both, then tested fixes.
Hold an LLM labeler fixed and swap who wrote its answer key: agreement rises when the key comes from a sibling model, and no human gold is needed to see it.
A fine-tuned encoder answers roughly 87% of customer calls. The rest go to an LLM with five candidate labels, and that shortlist needs a prompt of its own.
A 1% call issue can be absent from the sample an LLM proposes topics from. Over-splitting each category found 234 subcategories; flat clustering found 111.
We rewrite each customer call once into short, tagged statements. An encoder gains from clearer text alone; weaker prompted readers also gain from selection.
