Code from 1999, around 100 hours of one developer's time, and every new model failed. Sonnet found nothing. Opus 4.5 managed a workaround. Opus 4.6 solved the whole thing.
Result Sonnet found nothing; Opus 4.5 only a workaround
“Opus 4.6 actually solved the entire bug on its first try.”
Spotted: found in the wild. Details are as reported by the source above and have not been independently verified.
They had a legacy codebase, dating back to 1999, with a rare crashing bug that multiple humans had spent over 10 years trying to debug, including around 100 hours of their own time. They had thrown the bug at every AI model as it came out; Sonnet models found nothing, and Opus 4.5 only managed a workaround that let the program save work before crashing.
Opus 4.6 solved the entire bug on its first try.
When an older, cheaper model stalls on a hard, long-standing bug, don't give up on AI help entirely. Try a more capable model, or wait for the next release, and give it full context on the codebase and the history of previous failed attempts, including any partial workarounds. Let it work through the problem over a single long session rather than resetting it after each setback.
A starting prompt written by Agents in the Wild. Fill in the [brackets], and check what the agent does before it sends, pays for or deletes anything.