Article
Outcome Bias and Data-Driven Decision-Making Under Uncertainty
In the past, I judged my decisions by outcomes. Over time, I recognised that many choices I once deemed "bad" only had an unfavourable impact. Today, I realise that what truly matters in a decision is not the outcome but its optimality given the circumstances and available information, or, more accurately, how closely the decision approximates the optimal one.
Risk Tolerance: Why a Losing Decision Can Be Optimal
How should we approach decision-making in marketing and advertising, especially under uncertainty? Can a decision with a negative outcome be more optimal than one with a positive outcome? Yes. Judging a decision by its result rather than its reasoning is called outcome bias.
During my experience in decision-making in marketing and optimising budgets for online advertising, I have discovered that specific days within seasonal campaigns present exceptional opportunities. However, capitalising on these opportunities requires embracing a higher level of risk than on ordinary days. It is impossible to rely on data from the previous day with certainty, and waiting a few hours for data on a given day means the irreversible loss of many opportunities.
Without divulging sensitive details, the essential lesson I have gathered is that accepting a higher level of risk in these cases is advisable. Rather than fixating on individual situations, I focus on a 70% probability of a positive outcome in these cases. With this probability in mind, each decision of this type can be considered suitable, regardless of the impact of a single decision. What is important here is the high probability that the profit of decisions with a positive effect will cover the losses. In other words, I am acting on expected value: across many such decisions, the average result is positive even when a single one fails.
The Law of Large Numbers and Expected Value
I strive to implement this mindset when making all managerial or investment decisions. My anchor in these uncertain waters is the weak law of large numbers. It states that as the number of trials increases, the average of the outcomes will likely converge to the expected value. Through the lens of probability, I can maintain a strategic perspective, ensuring that, for example, my approach to online advertising optimisation remains resilient in the face of uncertainty.
The Gambler's Fallacy
The law of large numbers is easy to misread. It does not promise that a bad run will soon be balanced by a good one. Believing that a losing streak makes a win overdue is the gambler's fallacy, and it is exactly the wrong lesson. The law says nothing about the next trial; it speaks only about the average across many. Each decision stands on its own probability, not on what came before it.
Regression to the Mean
A related but distinct pattern is regression to the mean. An exceptional result rarely repeats. A campaign day that vastly outperforms, or one that collapses, is usually part luck, and the next result tends to drift back toward the average. It is why I try not to over-react to a single outcome, good or bad, and instead ask whether the process behind it was sound.
After reading this, it may seem like a simple thing. But answer this: how many people around you judge their steps by results alone?
Continue reading at: I make mistake after mistake
Summary
Common questions on this article's topic
Should a decision be judged by its outcome or by its quality?
What is the Weak Law of Large Numbers and how does it apply to business decisions?
How should you make decisions in marketing when data is incomplete?
What does it mean to be process-oriented rather than result-oriented?
Can a decision with a negative outcome still be the right decision?
Why do most people struggle with decision-making under uncertainty?
What is outcome bias?
What is expected value?
What is the gambler's fallacy?
What is regression to the mean?
What is thinking in bets?
What is decision theory?
How can you make better decisions under uncertainty?
If you have any thoughts, questions, or feedback, feel free to drop me a message at mail@richardgolian.com.
LinkedIn
Related articles
Are you a content creator playing by the rules of YouTube, Instagram or Google? Are you a small business owner whose business model depends on how often Amazon recommends you, how expensive Facebook ad impressions will be or how expensive Google Ads clicks will be? If all or most of your contact with your audience or customers depends on a big platform showing you to them, you are not in a strong position.
He used words like “certainty” as if statistics were part of Newtonian physics.
Yesterday I could not tear myself away from the computer. When I lifted my head, it was half past eight in the evening. I had been sitting alone upstairs for about three hours.
More articles
The development of artificial intelligence changes how we look at which paths in life make sense. Which studies and which profession to choose.
It was Thursday, 31 March 2022, when I handed over the last of my responsibilities to my colleagues, closed my laptop and went to the library. I left my job for several months and immersed myself in philosophy. And I am immensely grateful for it. Today I am, like most people, on a merry-go-round, and it is hard for me to stop and think about the fundamental questions of my own life.
It was September 2023. My life, my work, everything was completely different from now. It was simpler. I owned almost nothing. I needed to film an advert, I went up to a floor inside the company where the product I needed for that advert was, and there I noticed her for the first time. She laughs at me when I describe it like this, but at that moment the universe told me, that is her.
How and why I am gradually leaving social media and the big platforms. When I read that Meta is testing a new way to charge for organic posts that link outside its platform, it confirms how right I was to become steadily less dependent on the big platforms and on social media. It does not mean I have left them completely. It means that whether or not I stay in touch with my readers depends on them less and less, and today almost not at all.
One of my family members described a firm that has its data in several different tools, and its employees still join it by hand, in spreadsheets. That is the ordinary state of firms. When somebody tells such people not to do it by hand and to use AI, they will not understand. Most firms do not even have the basic connectors between the tools they use. These firms need help with AI transformation.
The same task, two models. Fable 5 against Opus 4.8. On paper Fable is the better model, with a larger context and stronger specs. And still it lost. Opus handled the task with a single round of checking for 721,000 tokens, while Fable needed nine rounds and burnt through 2.78 million tokens. The difference was not in the model, but in how I set the task. And I know it, because I measured it.
A few weeks ago I installed a small local AI model on my laptop that watches a live camera feed. I turned the webcam on in the dark, and in near total darkness it recognised me and the objects in the room. That such things exist, I have known for a long time. What opened my eyes was the accessibility. I installed it in one prompt, free, and it runs entirely on my machine, sending data nowhere.
I once wrote about building my own privacy-friendly analytics tool. It had bot detection from the first version, yet it was not enough. Direct visits took a strangely high share of my traffic. When someone claims that 20% of their visits are bots and 80% are humans, I used to think the same. Today I would say the opposite ratio is closer to the truth. This is how I got there.
I have Heidegger and my notebook beside me. I am asking where all of this is heading, where artificial intelligence is taking us.
Four days in Catalonia. No computer, no AI, almost no social media. I bought this notebook so that I could write down what I would think about, and what I would come across and learn on the trip.