> On an early checkpoint of LFM2.5-2.6B, 10.2% of completions on hard math and coding prompts produced repetitive loops. After Antidoom training, that rate fell to 1.4%, with eval scores improving across the board as a direct result of reduced looping.

That's a 9% reduction, and it doesn't really discuss applicability over longer contexts, different language/language familes/programming, or context/rope length aspects.

Several families - esp Llama and gemma - seem to be easily doom-inclined during expansions (I realize this is a pathological misuse of the tooling, but it's what people do): prompt: "add comments to this file", +1000t file read, [+3k reasoning], +800t edits, +1100t file re-read promoted by tool, prompt:"pep8 format it", [+2kt reasoning], +700t edits, +950t file re-read.

The unmodified/repeating sequences in those file reads, the differences between them, is a honeypot for attention.

I'd non-scientifically concluded for myself that this was tied to an anti-pattern for rope. In the first portion of the context inference needs attention to fall in certain places to produce the correct suggestions for the first round of edits, and somehow largely refocus attention to the subsequent version of the file in the context to correctly apply the subsequent edits, not getting distracted by the tantalizing pattern of repeated sequences and disrupted sequences

Using Gemma4 e2b I can repro this in LM-Studio, and fix it by hand-editing the context so that there's only the latest version of the file being worked on up at the start of the context, or replacing the original reads with something like:

" [tool name='read-file' path=...']... elided, see file.v1.py if you genuinely need to revisit the original state...[/tool] "

The handful of times I've tried this over the last 3 years it hasn't cared to discover file.v1.py didnt exist, limited samples = limited confidence.

It would be great if this could be combined with quantization-aware finetuning. In my experience, Qwen3.6-27B has much fewer repetitions at Q6 quantization level as compared to e.g. Q4, but that leaves little space for context on my 24GB RTX 3090.

I'm curious if this approach can be generalized beyond "doom loops".

For instance, another way of thinking about a "doom loop" is wasted tokens, which happens all the time with larger models that are inefficient at test time. Can "bad-ish" tokens be identified and penalized?

Maybe this is already SOTA but would love to learn more!

this is pretty cool. i think part of the root cause is current rlhf post training design around confidence and optics rather than cooperative transparent honesty. though its kinda an expensive hypothesis to dig into as a private individual

[flagged]