Fable 5 vs Opus: the cheaper model won

The same task. Two models. Fable 5 against Opus 4.8.

Who won? Opus. It handled the task with a single round of checking for 721,000 tokens, while Fable needed nine rounds and burnt through 2.78 million tokens.

On paper Fable 5 is the better model, so why did it lose?

"It's not the plane, it's the pilot."

The better prompt won. And I know it, because I measured it.

Fable 5 went in circles and burnt almost 4x more tokens.

Nine rounds

I commissioned a detailed audit of how data is processed on one project. It was a document full of hard facts: how many times an hour something runs, when it runs, which line of code it sits on. More than a hundred such claims.

And here is the catch. Not one of those numbers can be guessed. Either the right file is opened and what is actually in it gets read, or it does not. An invented fact looks just as convincing on paper as a true one. The difference shows only when someone actually checks.

One version of the report was prepared by Fable 5. To catch the mistakes, I put checkers on it. Not people, more AI. They had a single task: find an error in the report. When they found one, it was fixed and the checkers went to work again. And again. The rule was: repeat until one whole round finds not a single error.

It took nine rounds before they delivered a result without errors. Thirty-six agents in total.

Why nine?

If I have the best model on the market, why did it take nine rounds?

The answer has three parts.

First: most of those rounds did not fix errors from the audit. They fixed the checkers' own errors. The AI that was meant to find mistakes made more of them than it found. Twice it reported that a certain process runs every five minutes. In reality it ran far less often. Twice it miscalculated the size of the data. Once it reported an error in a file it was looking for in the wrong place.

When the rule is "repeat until it is flawless" and those doing the searching are often wrong themselves, the last rounds are not spent fixing. They are spent proving that the alarm was false.

Fable 5 was not disciplined, it cut corners

Second: this task was not about intelligence. It was about discipline in checking. The answers were not in the model's head, they were in the files. And even the cleverest model will not tell you what is in a file until it opens it and reads it. The model was not disciplined.

Third. Down to prompting skill.

Check before you write

The core is one sentence. Fact-checking belongs at the input, not the output.

Fable 5 did it like this: a subagent finds a fact, Fable takes it and writes it into the document. Verification came only at the end, on the finished text. It should have been the other way round. A fact is verified in the file before it ever reaches the document.

A plain table was missing too: each claim, its source, and who verified it. Without it, the checkers in every round looked at different parts of the document at random, so nothing was ever covered all at once. The last unverified claims therefore took the longest. With a table, one honest pass would cover everything and it would be done. Without it, it was a lottery.

It is a missing rule about where and when checking happens. And anyone can set that rule when they write the prompt.

What those nine rounds actually cost

I have exact figures from each round.