An A/B test hypothesis template is a fixed sentence structure that forces every experiment to state what will change, for whom, what result you expect and why. Teams that skip it end up testing button colors at random and can’t explain their winners. If you have followed our step-by-step guide to setting up an A/B test, the hypothesis is step one, and it decides whether the rest of the work teaches you anything.
This article gives you a copy-ready template, explains the null hypothesis behind every test, and provides 12 ecommerce examples you can adapt.
Before you start: where hypotheses come from
A strong hypothesis is based on evidence, not opinion. Collect at least one of these before writing:
- Analytics: a page or step with an unusually high drop-off, such as 70% of mobile visitors leaving the cart.
- Qualitative research: session recordings, heatmaps, on-site polls and support tickets that explain why visitors leave.
- Customer language: reviews and survey answers that reveal doubts (“I wasn’t sure about sizing”).
- Previous tests: results from your own test log or from similar stores.
The evidence goes into the “because” part of the template. Without it, you have a guess.
1. The A/B test hypothesis template
Copy this format:
If we [change] for [audience] on [page], then [primary metric] will [increase/decrease] by [at least X%], because [evidence and mechanism]. We will also watch [guardrail metric].
Each part has a job:
- Change: one specific, buildable change. “Improve the product page” is not a change. “Move delivery date under the add-to-cart button” is.
- Audience and page: where the test runs and who is included, such as mobile visitors to product pages.
- Primary metric: the single metric that decides the test, defined before launch.
- Expected effect: the minimum detectable effect you planned the sample size for.
- Because: the user behavior that explains why the change should work.
- Guardrail: the metric that must not get worse, such as return rate or page speed.
2. AB test hypothesis format: the short version
Some teams prefer a shorter AB test hypothesis format for test boards and tickets:
Because we saw [evidence], we believe [change] will cause [effect] for [audience]. We’ll know it worked when [metric] moves by [X%] after [N visitors per variant].
Both formats work. The important part is that the metric, the effect size and the reason are written before launch, so the result can’t be reinterpreted later.
3. The null hypothesis in A/B testing
Behind every business hypothesis is a statistical one. The null hypothesis (H0) states that there is no difference between control and variant: the conversion rate of B equals the conversion rate of A. The alternative hypothesis (H1) states that there is a difference.
The test doesn’t prove your business hypothesis. It calculates how likely the observed difference would be if H0 were true. If that probability, the p-value, falls below your significance level (usually 0.05), you reject the null hypothesis. If it doesn’t, you fail to reject it, which is not the same as proving the variant had no effect.
This is why A/B testing and hypothesis testing are closely linked: an A/B test is a randomized experiment analyzed with a statistical hypothesis test, usually a two-proportion z-test or a t-test.
4. Hypothesis for A/B testing: weak vs. strong
| Weak hypothesis | Strong hypothesis |
| A green button will do better. | If we raise contrast on the mobile add-to-cart button, add-to-cart rate will rise by 5%, because heatmaps show 40% of taps land below it. |
| Reviews will help. | If we show star rating and review count under the product title, product page conversion will rise by 8%, because 22% of exit-poll answers mention trust. |
| Let’s test free shipping. | If we show a cart progress bar to the $60 threshold, the share of orders over $60 will rise by 10%, because 30% of carts sit between $45 and $60. |
Read more: The Psychology Behind A/B Testing: How Small Changes Impact User Behavior
5. 12 A/B test hypothesis examples for ecommerce
Product page
- 1. If we add on-model images as the first gallery image, add-to-cart rate will increase by 6%, because product reviews mention that items look different in real life.
- 2. If we show the estimated delivery date under the add-to-cart button, conversion will increase by 4%, because exit polls show shipping time is the top unanswered question.
- 3. If we move photo reviews above the description, add-to-cart rate will increase by 5%, because scroll maps show only 35% of visitors reach the reviews.
Collection and search
- 1. If we add a “Best sellers” sort as the default, product page click-through will increase by 8%, because new visitors don’t know which items to open first.
- 2. If we show price and rating on mobile collection cards, click-through will increase by 5%, because visitors open and close several products to compare prices.
Cart and checkout
- 1. If we add a free shipping progress bar to the cart, units per order will increase by 7%, because 30% of carts are within $15 of the threshold.
- 2. If we enable express wallets above the checkout form, checkout completion will increase by 5%, because 60% of mobile checkout exits happen on the address form.
- 3. If we show the total with taxes in the cart, checkout abandonment will fall by 3%, because support tickets mention surprise costs.
Turn Your Hypotheses Into Live Tests
Build and launch the variant behind each hypothesis without writing code. Use every CustomFit.ai feature free for 14 days.
Offers and pricing
- 1. If we offer a bundle of three at 10% off, AOV will increase by 8% without lowering conversion by more than 2%, because 25% of orders already contain two or more units of the same item.
- 2. If we frame a subscription as “save 15% on every order”, the subscription take rate will increase by 10%, because survey responses show customers reorder monthly.
Landing pages and email traffic
- 1. If we match the landing page headline to the ad copy, bounce rate will fall by 10%, because visitors from paid social leave within five seconds.
- 2. If we replace the generic homepage with a category page for email traffic, revenue per visitor will increase by 6%, because email clicks are tied to a specific category.
Each example has a change, a metric, an effect size and a reason. That is enough to estimate the sample size and decide in advance what counts as a win. More examples, organized by funnel stage, are in this guide to writing A/B testing hypotheses.
6. How to prioritize hypotheses
When the list grows, score each hypothesis before building it. The ICE method rates Impact, Confidence and Ease from 1 to 10. The PIE method rates Potential, Importance and Ease. Multiply or average the scores and run the highest first.
Confidence should come from evidence. A hypothesis backed by analytics and three sources of qualitative research scores higher than one based on a competitor’s design. Keep every hypothesis, tested or not, in a shared library so ideas are not lost. The structure of such a library is explained in this article on building an A/B test hypothesis library.
7. Recording the result
After the test, add the outcome to the same document as the hypothesis:
- Result: win, loss or inconclusive, with the observed lift and confidence interval.
- Was the mechanism confirmed? For example, did scroll depth to reviews actually rise?
- What will you test next because of this result?
A losing test with a clear mechanism is valuable. It tells you what your customers don’t care about, which is often as useful as a win. For idea starters by vertical, see our list of A/B test ideas for FMCG and grocery brands.
8. One hypothesis from evidence to decision
Here is the full path for example 2 above. Evidence: an exit poll on product pages shows that 31% of answers ask “when will it arrive?”. Hypothesis: if we show the estimated delivery date under the add-to-cart button for all visitors to product pages, conversion will increase by at least 4%, because delivery time is the top unanswered question. Guardrail: late-delivery complaints.
The store converts at 2.5% and gets 9,000 product page visitors a day. Detecting a 4% relative lift would need far more traffic than the store has, so the team raises the expected effect to 10% and confirms it with research before launch. The test runs for three full weeks. Conversion rises 7% with a confidence interval of +1% to +13%, so H0 is rejected, and the follow-up hypothesis tests a “Get it by Friday” countdown. Choosing the right primary metric for each step is covered in our guide to choosing A/B testing metrics.
Hypothesis checklist
- One change per hypothesis.
- Evidence written in the “because” clause.
- One primary metric and at least one guardrail.
- Expected effect size that matches the sample size plan.
- Audience, page and device clearly stated.
- Written and shared before the test starts.
Pick One Hypothesis and Start Testing
Start with the hypothesis that has the strongest evidence. Try CustomFit.ai free for 14 days, cancel anytime.
Where to start
Take your top three ideas and rewrite them with the template above. If you can’t fill in the “because” clause, spend a week on research before testing. If you can, estimate the sample size for each and run the one with the best evidence and an achievable duration.
Summary
A good A/B test hypothesis follows a fixed template: if we make this change for this audience, this metric will move by this much, because of this evidence. Behind it sits the null hypothesis of no difference, which the statistical test tries to reject. Write the hypothesis before launch, record the result after, and keep both in a shared library.
FAQs
What is a hypothesis in A/B testing?
An A/B testing hypothesis is a testable prediction that states what you will change, which metric will move, by how much and why. For example: if we show delivery dates under the add-to-cart button, conversion will rise by 4%, because shipping time is the top unanswered question.
What is a good A/B test hypothesis template?
Use: “If we [change] for [audience] on [page], then [metric] will [increase/decrease] by [X%], because [evidence]. We will also watch [guardrail].” It forces you to define one change, one primary metric, an expected effect size and the reason behind it before launch.
What is the null hypothesis in an A/B test?
The null hypothesis states that there is no difference between control and variant, for example that both have the same conversion rate. The statistical test checks how likely your observed difference would be if that were true; a p-value below 0.05 lets you reject it.
Is A/B testing the same as hypothesis testing?
Not exactly. A/B testing is the experiment: randomly splitting users between versions. Hypothesis testing is the statistical method used to analyze it, such as a two-proportion z-test. Every A/B test relies on hypothesis testing to decide whether a difference is real or random.
How many hypotheses should I test at once?
Test one hypothesis per experiment on the same page and audience, so you know which change caused the result. You can run separate tests on different pages at the same time if their audiences and metrics don’t interact strongly.
