.jpg)
Share this article:
You Are Probably Doing the Math Wrong
Most people add their test results together. Three wins at 5 percent each? That is 15 percent total. Simple, satisfying, and wrong. A/B test gains are multiplicative, not additive. Each win applies to the new baseline the last win created. A 5 percent conversion rate that improves by 5 percent becomes 5.25 percent. The next 5 percent win lifts it to 5.5125 percent, not 5.5. The difference seems trivial at first. It is not. After 12 wins at 5 percent each, additive math says 60 percent improvement. The real number is 79.6 percent. After 20 wins, additive says 100 percent. Reality says 165 percent.
Don’t Ignore The Little Wins
A 4 percent lift on your pricing page will not make the quarterly all-hands. A 2 percent improvement in cart completion will not get you a promotion. So teams celebrate the big wins, ignore the small ones, and never realize the small ones were doing most of the work. This is exactly how compound interest operates. One year of 5 percent returns on a savings account is unremarkable. Twenty years is life-changing. The math is identical in conversion rate optimization (CRO). The discipline is in sticking with testing long enough for the multiplication to become impossible to ignore.
Most Tests Lose
Here is the part that scares people away: only about one in three A/B tests produces a winner. Two-thirds come back inconclusive or negative. That sounds discouraging until you do the math. Run 50 tests a year. Roughly 17 win. Give each winner an average lift of 4 percent. After one year, those 17 wins compound to a 95 percent cumulative improvement, not 68 percent. After two years with the same pace, you have more than tripled your baseline. The win rate is just a variable. It does not break compounding. It just sets the pace.
Losing Tests Are Not Wasted
Every loss narrows the field and sharpens your next hypothesis. Over time, this creates a compounding loop of its own: more tests produce more learnings, better learnings produce smarter hypotheses, smarter hypotheses produce higher win rates, and higher win rates accelerate the compounding of your actual conversion gains. The organizations that compound fastest are not the luckiest. They are the ones that learn the most from what did not work.
The Proof at Scale
Booking.com runs thousands of concurrent experiments. Virtually every product change is tested before it ships. The result is not one breakthrough feature that made the company. It is thousands of small, validated improvements stacked on top of each other over years. Microsoft runs tens of thousands of experiments annually across Bing, Office, and Xbox. Ronny Kohavi, who built their experimentation platform, has said repeatedly that roughly two-thirds of ideas fail when tested. The value comes from the one-third that wins, compounding over time. Google, Amazon, and Netflix follow the same playbook. None of them found a silver bullet. They all built systems that compound.
Speed Is the Multiplier
The fastest way to compound more is to test more. If one in three tests wins, then 12 tests a year produces 4 winners. Fifty-two tests a year produce 17. A hundred produces 33. The compounding difference is staggering. Four winners per year at 4 percent each compounds to about 60 percent over three years. Thirty-three winners per year at the same lift compounds to over 1,300 percent. That is why the best experimentation teams obsess over cycle time. Faster setup, faster QA, faster analysis. Every week you shave off the testing cycle is another compounding opportunity you gain over your competitors.
But Speed Without Aim Is Just Noise
Running a hundred tests on button colors will not compound into anything meaningful. The lift per winner matters as much as the number of winners. Twenty wins at 6 percent each compounds to a 221 percent improvement. Twenty wins at 2 percent each compounds to 49 percent. Same number of wins. Wildly different outcomes. The highest-compounding programs test revenue drivers: checkout flows, pricing pages, value propositions, and onboarding sequences. They build hypotheses from data, not opinions. Analytics, session recordings, customer support tickets, and user research all feed the hypothesis pipeline. Test big, meaningful changes on high-impact pages. Save the button colors for a slow Tuesday.
Four Ways to Kill Your Own Compounding
Not shipping winners. A winning variation sitting inside your testing tool is not an improvement. It is a temporary experiment that will eventually get removed. Until the win is hardcoded into production, it does not count toward compounding.
Redesigning from scratch. A team spends 18 months testing and validating dozens of improvements. Then someone launches a complete redesign that ignores it all. Test the new design against the optimized version, not the original.
Testing quarterly instead of continuously. Four tests a year means maybe one or two winners. You cannot compound one or two wins into anything meaningful. Compounding needs volume.
Not writing anything down. If nobody records what you tested, what you learned, and why the losers lost, every test starts from zero. Documentation turns disconnected experiments into a knowledge base that makes each subsequent test faster and smarter.
The Hidden Compound: What Your Team Knows
The conversion rate is the visible compound. Every test - win or lose - teaches your team something about your customers. What motivates them. What confuses them. What they skip entirely. That knowledge sharpens your instincts, improves your hypotheses, and raises your win rate over time. A team that has run 500 tests understands its audience in a way that no amount of market research can replicate. That understanding is a compounding asset, and your competitors cannot copy it by simply buying the same tools you use.
The Long Game
A/B testing gets discussed as a tactic, but that framing misses the point entirely. When you understand compounding, testing stops being a tactic and becomes a strategy. It is one of the rare growth levers that gets more powerful the longer you pull it. The companies that dominate their categories did not get there with one brilliant idea. They got there by running thousands of experiments, shipping hundreds of winners, and letting those gains compound over years.
The math behind compounding works the same whether you're running ten tests a year or a thousand. The difference is whether you have the infrastructure to keep the loop running. If you want to start compounding but aren't sure where to begin, or you've been testing sporadically and want to turn it into a real program, we can help you build the pipeline.
Talk to us about building a compounding CRO program.
