Dunning-Kruger Effect: beginners overestimate, experts underestimate

What you'll understand by the end of this lesson

• Why low-knowledge stakeholders make confident decisions while experienced practitioners express doubt
• How HiPPO-driven decisions reflect Dunning-Kruger Effect in organisational settings
• Why experienced CRO practitioners should speak up about what they know — even without certainty
• How to build a culture where decisions are made on evidence, not authority or false confidence

---

The principle in plain English

People with limited knowledge in an area tend to overestimate how well they understand it. They don't know what they don't know, so the gaps in their understanding aren't visible to them.

People with deep expertise, on the other hand, know exactly how much they don't know — the field is large, edge cases are everywhere, and certainty requires more evidence than they usually have. This makes them more cautious, more qualified in their claims, and sometimes more hesitant than they should be.

This is the Dunning-Kruger Effect, named after psychologists David Dunning and Justin Kruger, who documented it in a 1999 study. The pattern: low competence correlates with overconfidence; high competence correlates with appropriate uncertainty.

Neither end of the curve is optimal for decision-making. The beginner makes confident wrong decisions. The expert may be too cautious to act.

---

A simple example

A marketing director with no CRO training looks at a landing page and says: "The button should be bigger — that's obviously why we're not converting."

A CRO specialist looks at the same page and says: "I'm not sure yet. We need to look at the session recordings, check whether mobile and desktop have different drop-off patterns, and understand what users are actually confused about before we test anything."

The director is confident. The specialist is uncertain. In a meeting room, the confident voice often wins — even though the director's confidence reflects a lack of understanding of what the question actually requires, and the specialist's uncertainty reflects genuine expertise.

---

HiPPO decisions and Dunning-Kruger

HiPPO stands for Highest Paid Person's Opinion — a common dynamic in organisations where the most senior person's intuition overrides data, testing, and specialist knowledge.

The HiPPO problem is partly a power problem. But it is also a Dunning-Kruger problem: senior people who have operated in business for decades often have strong intuitions about customers. Some of those intuitions are correct. Many are not. The danger is that they cannot easily distinguish between the two — because the breadth of their experience gives them confidence in areas where they lack specific expertise.

A CEO who has run successful businesses knows a lot. They may know very little about the specific mechanics of conversion rate optimisation. But the confidence generated by a successful career makes it hard for them to model their own ignorance in a specialist area they haven't studied.

---

Expert paralysis: the other side of the curve

The Dunning-Kruger Effect describes both ends of the competence spectrum. The beginner problem is overconfidence. The expert problem is under-confidence — what is sometimes called "expert paralysis."

A CRO practitioner who has run hundreds of tests knows that conversion is context-dependent, that what worked elsewhere may not work here, and that certainty requires more data than is usually available. This is accurate. But it can manifest as reluctance to make recommendations, over-qualification of findings, and difficulty communicating clear guidance to stakeholders.

If every finding comes with a paragraph of caveats and every recommendation is conditional on five other variables, the expertise becomes inaccessible. Stakeholders stop listening. The HiPPO fills the void.

The experienced practitioner's challenge is to communicate with appropriate confidence — not the false confidence of the beginner, but not the paralysing uncertainty of the over-qualified expert either. The right output is a clear hypothesis with stated confidence and visible evidence: "Based on what we know, this is the most likely cause. Here's the test we'd run to confirm it."

---

Building a culture of evidence-based decisions

The structural response to Dunning-Kruger Effect in a CRO team is to make the process visible:

Document hypotheses before tests. When anyone — including the HiPPO — proposes a change, the expectation is that it comes with a stated reason and a predicted effect. This forces the person proposing the change to articulate their reasoning, which often surfaces how thin it is.

Attribute claims to evidence, not authority. "The data shows X" and "I believe X" are different claims. Making this distinction explicit — in reports, in meetings, in documentation — creates a culture where confidence is earned through evidence, not granted by position.

Celebrate negative test results. A test that disproves a hypothesis is useful data. Treating it as a failure (especially if the hypothesis came from a senior person) creates incentives to hide negative results. Treating it as learning creates incentives to test more.

---

The CRO audit

Look at how decisions get made in your organisation and ask:

1. Who is the final decision-maker on conversion changes?

If it's a senior non-specialist, Dunning-Kruger is a live risk. What process exists to ensure that specialist knowledge enters the decision, even when it conflicts with the senior person's intuition?

2. Are CRO recommendations expressed with appropriate confidence?

Review your last three reports or recommendations. Do they express clear guidance, or are they so heavily qualified that the stakeholder reading them doesn't know what to do? Appropriate confidence is not certainty — it's "based on the evidence, this is the direction, here's how we test it."

3. Are negative test results documented?

If your team's test log only contains wins, there may be a cultural pressure to not document — or not run — tests that challenge confident hypotheses from senior people. A healthy log contains all results.

---

---