Article
Fable 5 vs Opus: the cheaper model won
The same task. Two models. Fable 5 against Opus 4.8.
Who won? Opus. It handled the task with a single round of checking for 721,000 tokens, while Fable needed nine rounds and burnt through 2.78 million tokens.
On paper Fable 5 is the better model, so why did it lose?
"It's not the plane, it's the pilot."
The better prompt won. And I know it, because I measured it.
Fable 5 went in circles and burnt almost 4x more tokens.
Nine rounds
I commissioned a detailed audit of how data is processed on one project. It was a document full of hard facts: how many times an hour something runs, when it runs, which line of code it sits on. More than a hundred such claims.
And here is the catch. Not one of those numbers can be guessed. Either the right file is opened and what is actually in it gets read, or it does not. An invented fact looks just as convincing on paper as a true one. The difference shows only when someone actually checks.
One version of the report was prepared by Fable 5. To catch the mistakes, I put checkers on it. Not people, more AI. They had a single task: find an error in the report. When they found one, it was fixed and the checkers went to work again. And again. The rule was: repeat until one whole round finds not a single error.
It took nine rounds before they delivered a result without errors. Thirty-six agents in total.
Why nine?
If I have the best model on the market, why did it take nine rounds?
The answer has three parts.
First: most of those rounds did not fix errors from the audit. They fixed the checkers' own errors. The AI that was meant to find mistakes made more of them than it found. Twice it reported that a certain process runs every five minutes. In reality it ran far less often. Twice it miscalculated the size of the data. Once it reported an error in a file it was looking for in the wrong place.
When the rule is "repeat until it is flawless" and those doing the searching are often wrong themselves, the last rounds are not spent fixing. They are spent proving that the alarm was false.
Fable 5 was not disciplined, it cut corners
Second: this task was not about intelligence. It was about discipline in checking. The answers were not in the model's head, they were in the files. And even the cleverest model will not tell you what is in a file until it opens it and reads it. The model was not disciplined.
Third. Down to prompting skill.
Check before you write
The core is one sentence. Fact-checking belongs at the input, not the output.
Fable 5 did it like this: a subagent finds a fact, Fable takes it and writes it into the document. Verification came only at the end, on the finished text. It should have been the other way round. A fact is verified in the file before it ever reaches the document.
A plain table was missing too: each claim, its source, and who verified it. Without it, the checkers in every round looked at different parts of the document at random, so nothing was ever covered all at once. The last unverified claims therefore took the longest. With a table, one honest pass would cover everything and it would be done. Without it, it was a lottery.
It is a missing rule about where and when checking happens. And anyone can set that rule when they write the prompt.
What those nine rounds actually cost
I have exact figures from each round.
Join the Library
Full access to my findings, personal stories, and what I learn from the people I meet.
Join the Library · €29.99 per year How a small business, creator or politician can be… What should I study in the age of AI?… I Left My Job for Several Months to Read… How I Fell in Love With Her, at First… How do you stop depending on social media? Can… How to Start With AI in a Company: Automating…Get the full article by email and feel free to reply if you want to discuss it further.
Summary
Common questions on this article's topic
Which AI model is better, Fable 5 or Opus 4.8?
Why does an AI make so many mistakes when checking facts?
How do you write a better prompt for an AI audit or review?
Does the model or the prompt matter more with AI?
How many tokens does an AI audit use?
What does checking facts at the input rather than the output mean?
If you have any thoughts, questions, or feedback, feel free to drop me a message at mail@richardgolian.com.
LinkedIn
Related articles
I see a lot of AI fatigue around me, loss of motivation and burnout. One example is a colleague who is leaving her job and says she plans to leave the digital marketing industry altogether.
Are you a content creator playing by the rules of YouTube, Instagram or Google? Are you a small business owner whose business model depends on how often Amazon recommends you, how expensive Facebook ad impressions will be or how expensive Google Ads clicks will be? If all or most of your contact with your audience or customers depends on a big platform showing you to them, you are not in a strong position.
Recently I have interviewed many candidates for well-paid roles: managers, team leaders and advertising specialists. I have to say that with most of them, the conversations would have made sense in 2024 or 2025, but not today.
More articles
The development of artificial intelligence changes how we look at which paths in life make sense. Which studies and which profession to choose.
It was Thursday, 31 March 2022, when I handed over the last of my responsibilities to my colleagues, closed my laptop and went to the library. I left my job for several months and immersed myself in philosophy. And I am immensely grateful for it. Today I am, like most people, on a merry-go-round, and it is hard for me to stop and think about the fundamental questions of my own life.
It was September 2023. My life, my work, everything was completely different from now. It was simpler. I owned almost nothing. I needed to film an advert, I went up to a floor inside the company where the product I needed for that advert was, and there I noticed her for the first time. She laughs at me when I describe it like this, but at that moment the universe told me, that is her.
How and why I am gradually leaving social media and the big platforms. When I read that Meta is testing a new way to charge for organic posts that link outside its platform, it confirms how right I was to become steadily less dependent on the big platforms and on social media. It does not mean I have left them completely. It means that whether or not I stay in touch with my readers depends on them less and less, and today almost not at all.
One of my family members described a firm that has its data in several different tools, and its employees still join it by hand, in spreadsheets. That is the ordinary state of firms. When somebody tells such people not to do it by hand and to use AI, they will not understand. Most firms do not even have the basic connectors between the tools they use. These firms need help with AI transformation.
A few weeks ago I installed a small local AI model on my laptop that watches a live camera feed. I turned the webcam on in the dark, and in near total darkness it recognised me and the objects in the room. That such things exist, I have known for a long time. What opened my eyes was the accessibility. I installed it in one prompt, free, and it runs entirely on my machine, sending data nowhere.
I once wrote about building my own privacy-friendly analytics tool. It had bot detection from the first version, yet it was not enough. Direct visits took a strangely high share of my traffic. When someone claims that 20% of their visits are bots and 80% are humans, I used to think the same. Today I would say the opposite ratio is closer to the truth. This is how I got there.
I have Heidegger and my notebook beside me. I am asking where all of this is heading, where artificial intelligence is taking us.
Seventy per cent. That is where the first AI output begins, even when you give it the full company context and the best examples from the past. We are talking about the kind of output that cannot be defined programmatically. It is more complex. Often it is creative work. On one repeated type of output I reached eighty per cent within a week. Every further percentage point is harder than the one before.
Four days in Catalonia. No computer, no AI, almost no social media. I bought this notebook so that I could write down what I would think about, and what I would come across and learn on the trip.