What you'll understand by the end of this lesson
- Why the source of a hypothesis determines its quality before a single test begins
- Seven categories of insight CRO practitioners use to replace guesswork with evidence
- What analytics can and cannot tell you about visitor behaviour
- How user testing, rapid online tests, and behaviour tools each contribute different types of evidence
- How to capture direct barrier data from customers who have already converted
Why the source of an idea matters
By the end of Lesson 2, you had a process: capture ideas, write hypotheses, rank with ICE. The next question is where the ideas actually come from.
This is more important than it sounds. The quality of a test is set before it is designed. A hypothesis built on what the team thinks would look better is a structured guess. A hypothesis built on observed visitor behaviour and documented customer concerns is something worth investing in.
The CRO term for inputs that generate well-evidenced ideas is sources of insight. There are seven categories. Used together, they make it possible to maintain a hypothesis backlog that is never based on guesswork alone.
Source 1: Published research and industry data
The fastest starting check is whether someone has already tested a similar direction and documented what happened.
This is not about copying best practices — there are none in CRO. What works for a travel booking site often fails on a B2B software platform. But a published case study that found a particular direction worked in a similar context does something useful: it raises your confidence before the test begins.
How to apply it: Before writing a hypothesis, spend 20 minutes searching for documented experiments in your category. If you find supporting evidence, raise the ICE confidence score. If you find that the direction consistently failed in similar contexts, lower the score or discard the idea.
Published evidence is a starting point, not a conclusion. Use it to calibrate confidence — not as a substitute for testing with your own visitors.
Source 2: Research you've already paid for
Most organisations have commissioned audience research at some point. Surveys. Persona documents. Recorded customer interviews. A focus group report. There was a presentation, everyone said they learned something, and the files went onto a shared drive.
That research is a hypothesis list waiting to be written.
Pull every piece of audience research your organisation holds and read it with one question: does the website currently address what this research says our audience needs? If a customer survey found that implementation time was buyers' primary concern, does the pricing page speak to that? Every gap between what the research documents and what the website communicates is a testable hypothesis.
These tend to score well on the ICE confidence dimension — the evidence is specific, documented, and came directly from real customers.
Source 3: Analytics
Analytics is your behavioural data layer. It shows you what visitors do: which pages they visit, which paths they take, where they leave, which funnel steps produce the most drop-off.
This is valuable for two specific purposes:
Locating where a problem exists. A sharp drop-off at the payment step, a high-traffic page with an unusually low conversion rate — analytics identifies these precisely. You know exactly where to focus.
Establishing the scale of a problem. A friction point on a page with 400 monthly visitors is a low-priority problem. The same friction point on a page with 40,000 monthly visitors is a fundamentally different conversation.
Analytics tells you what is happening, not why. A 68% checkout abandonment rate confirms there's a problem — it doesn't tell you whether the cause is unexpected costs, confusing field labels, or missing trust signals. Pair analytics findings with at least one qualitative source before writing a hypothesis.
Source 4: User testing
User testing means watching a real person use your website while they narrate what they're doing and thinking.
The insight is qualitative and specific. You see where someone hesitates. You hear what they looked for and failed to find. You observe the moment a piece of copy confused them. These observations aren't statistically representative — eight sessions can't tell you what percentage of all visitors experience a given problem — but they're essential for generating hypotheses that are precise and grounded in actual user experience.
What good user testing looks like:
- Give participants a realistic task, not a vague instruction ("try to sign up for the free trial" rather than "explore the website")
- Ask them to think aloud — narrate what they see and what they're uncertain about
- Watch without intervening, even when they struggle. The struggle is the data.
- By the third or fourth session, patterns start to appear. By eight, the most common issues are usually clear.
If you have never watched a real person try to complete a purchase on your site, do it before you write your next hypothesis. It consistently reveals problems that analytics would never surface.
Source 5: Rapid online tests
Between the qualitative depth of user testing and the statistical rigour of an A/B test, there's a range of lightweight methods that can sharpen a hypothesis quickly.
First click test — Participants are shown a page and asked where they'd click to complete a specific task. Results show what percentage clicked the intended target and what they clicked instead. Particularly useful for evaluating whether a CTA is visible and correctly labelled.
Five-second test — A design is shown for five seconds, then hidden. Participants answer: what does this company do? What is the main offer? If most participants can't answer after five seconds, the headline and hero are not doing their job.
These methods run with 25–50 participants and typically complete within hours. They're not A/B tests — they won't tell you which version converts better statistically. What they give you is direction and confidence before you invest in the full experiment.
Source 6: Behaviour analytics tools
Where user testing shows one person's experience in depth, behaviour tools show aggregate patterns across thousands of sessions.
Heat maps and click maps — Visualise where visitors click, tap, or move across a page. Typical findings: clicks on images that aren't links (visitors expected interactivity); clicks on navigation items the team didn't anticipate (visitors are looking for something they can't find elsewhere). Each finding is a hypothesis.
Scroll maps — Show what percentage of visitors reach each point on a page. If 65% of visitors stop scrolling before they reach your social proof section, that section is being seen by 35% of your audience — not 100%. That changes whether you improve the content or simply move it higher.
Session recordings — Replay individual visits. Most useful when filtered to specific behaviours: visitors who abandoned checkout, or who spent several minutes on the pricing page without converting.
Behaviour tools run passively once installed. Start collecting data early — even if you don't review it immediately, the patterns compound over time and become more useful as traffic accumulates.
Source 7: Post-conversion feedback
When a visitor completes your conversion — buys, subscribes, signs up — they've just made a decision and remember what nearly stopped them.
Ask them one question, immediately after conversion:
"What almost stopped you from [buying / signing up / subscribing]?"
This is counterintuitive. You're asking a satisfied customer to recall their hesitation. It works because having just chosen something, they answer honestly about what almost stopped them — they're describing a problem they solved, not criticising a business they regret.
Running this effectively:
- Ask one question, not a survey. Response rates drop sharply with every additional question.
- Ask at the moment of conversion: on the confirmation page, or in the immediate post-purchase email.
- Keep the question consistent over time — patterns across hundreds of responses are as valuable as any individual answer.
How the sources work together
No single source is sufficient alone. Analytics locates problems but can't explain them. User testing produces insights that may not correspond to where the real volume is. Post-conversion feedback reveals what barriers exist but not how many visitors encounter them.
The strongest hypotheses draw on more than one source. A typical evidence chain:
- Analytics shows a drop-off at the pricing page
- Scroll map reveals most visitors stop before the comparison section
- Post-conversion feedback shows "I almost left because I couldn't find pricing for smaller teams"
- Hypothesis: If I add a small-team pricing note in the first visible section of the pricing page, I expect time-on-page and conversion from pricing to increase.
That hypothesis is built on three independent sources pointing in the same direction. It is not a guess.
| Source | What it gives you | Best used for |
|---|---|---|
| Published research | Prior confidence and direction | Calibrating ICE before writing |
| Existing research | Documented customer concerns | High-confidence ideas at no cost |
| Analytics | Location and scale of the problem | Finding where to focus |
| User testing | Qualitative depth, specific causes | Understanding why analytics looks the way it does |
| Rapid online tests | Directional evidence, fast | Sharpening hypotheses before full experiments |
| Behaviour tools | Aggregate patterns across real sessions | Quantifying behaviour analytics can't capture |
| Post-conversion feedback | Real barriers, from people who overcame them | The most direct evidence of what visitors actually encounter |
Analytics tells you a page has a 72% drop-off rate. What does analytics alone NOT tell you?
You've gathered evidence and written a strong hypothesis — now how do you actually test it in a way that produces data you can trust?