- Why having too many ideas is just as dangerous as having none
- The one-sentence formula that turns a vague idea into a testable hypothesis
- How to rank your ideas so the right ones get tested first
Every team working on a website has ideas. The designer wants to simplify the layout. The marketer wants a stronger headline. The CEO just saw a competitor's site and wants to know why it looks more modern. The customer success team keeps hearing the same question on every call and suspects the homepage isn't answering it.
Ideas are not the problem. The problem is that without a system, ideas are just noise — a constant stream of opinions competing for priority, with no shared way to evaluate them.
CRO is often associated with A/B testing tools and statistics. But before any of that matters, there's a quieter discipline that determines whether a CRO program actually produces results: idea management. For the deeper version of this lesson, see Managing ideas and hypotheses in the CRO learning path.
The list that holds everything together
The starting point is simpler than most people expect. You need a list — a spreadsheet, a Notion table, a document. Something that can hold every idea that surfaces, with a note on who suggested it and what part of the site or campaign it addresses.
That's it. No special software, no elaborate tagging system. A place where ideas land instead of getting lost.
The list does two things. First, it becomes a resource you return to when you're looking for what to work on next — a searchable, sortable backlog of possibilities. Second, and less obviously, it becomes a professional shield.
When someone walks into your workspace with an idea they heard from a competitor or saw on LinkedIn, you don't have to argue with it, dismiss it, or immediately drop what you're doing. You add it to the list, commit to researching it, and move on. "Great idea — it's on the list" is a complete response that respects the suggestion without letting it hijack your roadmap.
The list isn't just an organizational tool — it's how you stay in control of your own priorities without having to say no to anyone directly.
The gap between an idea and a hypothesis
Once an idea is on the list, it needs to earn its place at the top. And the first test it has to pass is simple: can you write a specific hypothesis around it?
Most ideas, when examined honestly, are vague. "We should refresh the website" is the classic example. Everyone nods. No one asks what "refresh" means. The idea makes it onto the roadmap, a team spends weeks on it, and then nobody can evaluate whether it worked — because success was never defined.
A hypothesis forces specificity. The format is:
If I [make this specific change], I expect [this specific outcome], as measured by [this specific metric].
Applied to the same vague idea:
- If I remove three form fields from the signup page, I expect the completion rate to increase, as measured by Google Analytics goal completions.
- If I add whitespace between sections on the product page, I expect users to scroll further down the page, as measured by average scroll depth.
- If I rewrite the hero headline to focus on the outcome rather than the feature, I expect click-through on the primary CTA to increase, as measured by GA4 click events.
Notice what happened. "Refresh the website" — one vague idea — became three specific, testable hypotheses. Each one names a change, a prediction, and a way to know if the prediction was right.
The hypothesis template doesn't just organize your ideas. It reveals which ones were actually ideas, and which ones were just wishes.
This is the multiplier effect of good hypothesis writing. One vague mandate becomes several concrete possibilities. Each one is now auditable — you can look at it and ask: do we have evidence that this problem exists? Is the proposed change actually addressing the right cause? Is our measurement capturing the right thing?
Ranking what's worth testing first
After hypotheses, the next step is prioritization. Not all ideas deserve equal attention, and some that look good on paper are genuinely hard to execute or unlikely to move the metric much.
The ICE framework is a practical way to rank your backlog. For each hypothesis, score three things from 1 to 5:
- Impact — if this works, how significantly will it affect the metric?
- Confidence — how much evidence already exists that this problem is real and this change will help?
- Effort — how much work does it take to build and run this test?
Add the three scores, sort highest to lowest. The ideas with the highest scores get tested first.
A few things worth knowing about ICE:
Consistency matters more than accuracy. You're not trying to predict the future — you're creating a shared framework for making relative comparisons. It's fine to give "change the hero headline" an effort score of 1 and "produce videos for all product pages" an effort score of 5 without knowing exactly how long each will take. The relative ranking is what matters.
Confidence grows with research. Start by doing a rough pass on all your hypotheses. Then go back and research the ones near the top — check your analytics, look at session recordings, review customer feedback. As your evidence grows, your confidence scores will change, and so will the ranking.
Low effort ideas float up. The ICE framework naturally surfaces quick wins — high-impact, low-effort hypotheses that are easy to test and likely to produce a clear result. This is where a new CRO program should start. Quick wins build momentum and generate the data that sharpens the bigger bets later.
ICE is a starting point, not a verdict. A hypothesis that scores low today might score much higher after you've done an hour of analytics research. The ranking is a living document, not a fixed list.
What this looks like in practice
Here's the flow from start to first test:
- Capture every idea as it surfaces — from data, from colleagues, from customers, from your own observation. Add it to the list without filtering.
- Write a hypothesis for each idea. Some will immediately reveal themselves as too vague to hypothesize around — that's useful information. Those ideas either get clarified or stay at the bottom of the list.
- Score on ICE — a quick, consistent first pass. Don't overthink the scores.
- Research the top ideas — look at analytics, recordings, customer interviews to see if the problem your hypothesis addresses is real and significant.
- Update the scores based on what you find. The highest-scoring, best-evidenced hypothesis becomes the next test.
This process doesn't require expensive tools or a full CRO team. It requires a spreadsheet, a hypothesis template, and the discipline to use them consistently. If you want to understand why the underlying instinct to "refresh" is so common — and so costly — read Why your gut feeling about your website is almost always wrong.