Claude Fable 5 Tops AI Coding Benchmark with 64% Success Rate
Taylor Wilson
Anthropic's Claude Fable 5 scored 64% on the MirrorCode programming benchmark, more than triple the 21% of runner-up GPT-5.6 Sol — a quantified signal that frontier AI models can now independently deliver mid-to-large software projects when goals are well-defined.
What does MirrorCode actually test?
MirrorCode is an extreme coding benchmark. The model is dropped into a sealed environment — no internet, no original source code, no third-party dependencies. All it gets is high-level documentation and a set of visible tests.
In plain terms = imagine locking a developer in a room with only a user manual and a few exam questions, then asking them to write an entirely new program that behaves identically to the original.
The bar is unforgiving: both visible and hidden tests must hit 100%. One missed edge case means failure. The latest run picked 15 mid-to-large targets from a pool of 25, each attempted in two programming languages with three runs apiece.
64% vs 21% — what does that gap mean?
Claude Fable 5 cleared 64% of targets; GPT-5.6 Sol managed roughly 21% — a gap of more than three-to-one.
This means → in engineering tasks with clear goals and auto-verifiable outcomes, Fable 5 reliably completes nearly two-thirds of mid-to-large projects, while competitors stay around one-fifth.
This reflects a gap between frontier models that is no longer fine-tuning-level — it is generational.
It still works in a rare language — why does that matter?
Fable 5 hit 64% in Go, a mainstream language. Switching to Ada — a niche language used mainly in aerospace and defense — it still scored 61%, dropping just 3 percentage points.
Ada's public training data is vanishingly small. StarCoder figures show Python at roughly 8% of the corpus; Ada sits at about 0.034% — Python has roughly 230× more data.
In plain terms = the model has barely seen Ada in training, yet performs almost as well as in Go. It is not pattern-matching memorized code; it is reasoning about program behavior at the logic level. By contrast, GPT-5.5 fell from 17% to 5% when the language switched — exposure-dependent performance laid bare.
A single task can cost $2,600 — is that worth it?
MirrorCode allows up to 10 billion tokens per attempt and permits runs as long as 7 days. The most expensive task on record ran for 19 days and cost $2,600.
A concrete case: gotree, a bioinformatics tool with roughly 16,000 lines of Go and over 40 commands. Claude Opus 4.7 finished in 14 hours for $251, passing 2,000 of 2,001 tests — 99.95% completion — failing only on one edge case involving date-comment parsing.
Epoch AI estimates a human engineer would need two to seventeen weeks for the same task. This means → even at the $2,600 extreme, the cost compares favorably to weeks of engineer salary for tasks that are well-defined and auto-verifiable.
What does this mean for commercial AI coding?
MirrorCode's 64% is a quantified proof point for "project-level AI delegation" — not writing a single function, but independently delivering a mid-to-large software project.
This means → under conditions of clear objectives and automatic acceptance testing, frontier models have moved past the "assistant tool" phase and begun to touch the threshold of independent delivery.
The open question remains: can this capability boundary hold in broader, more ambiguous engineering contexts? The path from 64% to commercially deployable reliability has only just begun.
Content is for reference only, not financial advice.