- Why brainstormed A/B test ideas fail most of the time — and the six sources that replace guesswork with real evidence
- The market research sitting on your hard drive that is almost certainly worth more than any new CRO tool
- The one question to ask customers the moment they convert — and why the answer beats a month of internal debate
Most CRO programs follow the same pattern: the team meets, someone says "I think we should try this," a few people nod, and the idea becomes a test. Three weeks later the result is flat. The team looks for the next idea.
The failure is not in the testing. It is in where the ideas are coming from.
A brainstorm produces ideas that make sense to the people in the room — people who know the product intimately, have aesthetic opinions shaped by months of iteration, and have lost the ability to experience the site the way a first-time visitor does. Every brainstormed idea reflects that insider perspective, which is exactly why so many of them fail to move any metric.
Good CRO ideas come from observation, not opinion. Here is where. (The CRO glossary defines the key terms used throughout.)
A test with no research behind it is just a guess with extra steps.
The research collecting dust on your hard drive
Most organisations have done some form of audience research. Surveys. Persona documents. A consultant ran focus groups. There was a presentation, everyone agreed they'd learned a lot, and the report went onto a shared drive and was never opened again.
That document is a hypothesis list.
If a focus group two years ago found that buyers' top concern was whether they could get implementation support, the question of whether your website currently answers that — and where in the funnel — is a testable hypothesis with documented evidence behind it. Read every piece of audience research your organisation holds with one question in mind: does the website currently answer what this research says our audience cares about? Every gap is an idea. These tend to score well on the ICE confidence dimension, because the evidence is specific and documented rather than assumed.
Analytics: a location tool, not a strategy tool
Analytics is indispensable for one job: finding where a problem exists and how large it is. A sharp drop-off at the shipping step, a high-traffic page with a low conversion rate, a form with a 78% abandonment rate — analytics pinpoints these with precision.
What it cannot do is explain why. Knowing that 72% of visitors abandon checkout at the payment step tells you where to look. It does not tell you whether the cause is unexpected costs, confusing form labeling, or missing trust signals. That requires a different source.
Use analytics to rank and validate problems. Use qualitative research to understand what to do about them.
Watching real people use your site
User testing — recruiting real people to attempt tasks on your website while narrating their thinking — is the fastest way to understand why analytics looks the way it does. Modern tools have made this far more accessible than it once was. A study of eight to twelve participants can be recruited, recorded, and reviewed within a day.
What you get is not statistically representative, but it is irreplaceable: direct observation of where someone hesitates, what they looked for and failed to find, and what language or element caused confusion. Every session typically surfaces two or three hypotheses you would not have generated any other way.
Rapid tests that sharpen your thinking before the real experiment
Two tools are worth knowing before you commit to a full A/B test:
First click test — Participants are shown a page and asked to click where they would go first to complete a specific task. If 65% of people click somewhere other than your intended CTA, that is a clarity problem — confirmed by data rather than argued about in a meeting.
Five-second test — A design is shown for five seconds, then hidden, and participants answer questions: what do you remember, what does this company do, what is the main offer? This tells you whether your headline and hero are communicating your value proposition in the time most visitors actually spend before deciding to stay.
Both tests run with 25–50 participants and complete in a few hours. They do not replace A/B testing — they make whatever hypothesis you take into an A/B test significantly sharper. For a full explanation of why A/B data is in a class of its own, see Why A/B testing is the most honest data you can collect.
Behaviour tools: patterns your analytics won't show you
Heat maps and scroll maps accumulate patterns across thousands of real sessions. A scroll map reveals what percentage of visitors reach each section of a page — if most people stop scrolling before your proof section, no amount of improving that section's copy will matter until you move it. A click map shows where people are clicking, including on elements that are not links, which reveals that visitors expected interactivity the page does not deliver.
A golf resort discovered this the hard way. Their features page led with golf — obviously the main thing, everyone agreed. Click data showed 19 clicks on golf and 176 on free breakfast. The team had spent months optimising the wrong section. One heat map session moved free breakfast to the top. No A/B test required to know which direction to test first.
Session recordings let you replay individual visits. Most useful when filtered to specific behaviours — visitors who abandoned checkout, visitors who spent several minutes on the pricing page without converting. Unfiltered, the volume is too high to learn from reliably.
The one question every converting customer should answer
Immediately after someone converts — buys, signs up, subscribes — ask them this:
"What almost stopped you?"
It sounds counterintuitive. You are asking a happy customer about their hesitation. But it works — and there is a reason why. Psychologists call it the liking effect: someone who has just chosen you feels positively toward you. They do not experience the question as criticism; they experience it as a chance to reflect on a decision they are proud of. That emotional state produces honest, specific answers at unusually high completion rates.
The answers tell you what real visitors are encountering in your conversion flow. Not the barriers your team hypothesised about in a meeting. The actual ones.
One question, not a survey. Asked immediately. Completion rates are high because the emotional context is right — and specific, actionable answers come back.
The pattern underneath all of it
Every source in this list has the same property: it is not your opinion. Analytics observes. User testing observes. Behaviour tools observe. Existing research documents what your audience told you directly. Rapid tests collect responses from people outside your building. Post-conversion feedback captures barriers from people who walked through them.
The shift from "we think this might work" to "here is what the evidence suggests, and here is what we predict" is not just a semantic one. If you want the full structured walkthrough of all seven sources with worked examples, the Understanding your visitors lesson covers each one in depth. It is the difference between a test that teaches you something regardless of outcome and a test that just confirms whatever you wanted to believe.