Running multiple AB tests at the same time is one of the first questions a growing store asks once the test backlog gets longer than the traffic. The worry is easy to understand. If shoppers see a new product page in one test and a new cart drawer in another, how do you know which change moved the numbers? The short answer is that overlapping tests are usually fine, as long as you know when they are not. This post looks at what really happens when tests share the same visitors.
Below we explain how concurrent tests work, how to spot an interaction effect, when to switch to mutually exclusive groups, and how the traffic math changes for small stores.
Running Multiple AB Tests at the Same Time: The Short Answer
Yes, you can run several tests at once on the same site, and most mature experimentation teams do. When each test assigns visitors at random and independently, the other tests act like background noise. A shopper in variant B of your product page test is equally likely to be in either arm of your checkout test, so the checkout test still compares like with like.
Randomization is what protects you. It does not remove the effect of the other test. It spreads that effect evenly across both arms, so it cancels out when you compare them. This is the same reason you can run a test during a sale or a holiday week: the sale affects both arms equally.
The real risk is narrower than most people expect. It only appears when two changes interact, which means the effect of one change depends on whether the shopper also saw the other change. We cover that case below, because it is the one that justifies extra caution.
Concurrent AB Tests vs Overlapping Experiments
People use these terms loosely, so it helps to separate them.
- Concurrent AB tests are any tests that run during the same period. They might touch different pages, different audiences or the same funnel.
- Overlapping experiments are concurrent tests where the same visitor can be in more than one test. This is the default on most testing tools unless you set up exclusion.
- Mutually exclusive experiments are concurrent tests where each visitor can enter only one test from a group, so no shopper ever sees two of those changes together.
A homepage hero test and a post-purchase survey test are concurrent, but they barely share a journey, so overlap is a non-issue. A product page price display test and a cart page discount message test share the exact same buyers at the moment of decision. That pair deserves a closer look.
Google described its approach to this problem in a well-known paper on overlapping experiment infrastructure. Tests are grouped into layers. Tests in different layers overlap freely, while tests in the same layer, which change the same parameters, are kept apart. You do not need Google’s scale to borrow the idea.
Run More Tests Without the Guesswork
Plan overlapping and exclusive tests on the same store, with random assignment handled for you. Try CustomFit.ai free for 14 days.
What Is an Interaction Effect in AB Testing?
An interaction effect happens when the lift from one change is different depending on the variant of another test. Here is a simple worked example with illustrative numbers. A store runs two tests on the same visitors:
- Test 1 changes the product page to show a free shipping message.
- Test 2 adds a free shipping progress bar to the cart.
Each visitor lands in one of four combinations. The conversion rates below are example data to show the pattern, not results from a real store.
| Combination | Product page | Cart | Conversion rate |
| A + A | Original | Original | 3.0% |
| B + A | Free shipping message | Original | 3.3% |
| A + B | Original | Progress bar | 3.3% |
| B + B | Free shipping message | Progress bar | 3.35% |
On their own, each change lifts conversion by about 10%. Together, they add almost nothing beyond either one alone, because both changes deliver the same message. The second reminder about free shipping adds little once the shopper already knows. This is a negative interaction.
What does each test report? Test 1 compares everyone who saw the message with everyone who did not, averaged across both cart versions. It shows a lift of roughly 5%, not 10%. The result is still a fair answer to the question “what does this change do on the site as it runs today, with the other test live?” It is just a blended answer.
Interactions can also be positive. A clearer size guide and a looser returns policy might each help a little and help much more together, because one removes doubt before purchase and the other removes risk after it.
How to Spot an Interaction Effect AB Testing Teams Miss
You do not need a statistics package to check for interactions in most cases. A quick review covers it.
- 1. Map the overlap. List every live test, the pages it touches and the metric it targets. Tests on unrelated pages with different goals almost never interact in a way that matters.
- 2. Look for shared messages or shared elements. Two tests that both promote free shipping, both change the price display, or both edit the same button are the usual suspects.
- 3. Break down one test by the other. Most tools let you segment results by the variant of another experiment. If Test 1 shows a strong lift in one arm of Test 2 and no lift in the other, you have an interaction.
- 4. Check for visual conflicts. Sometimes the problem is not statistical at all. Two tests that inject banners into the same slot can break the layout for shoppers who get both variants. QA every combination before launch.
Keep the segment check in proportion. Splitting a test into smaller groups cuts its power, so small differences between segments are usually noise. Look for large, consistent gaps, ideally ones with a clear reason behind them.
Microsoft’s experimentation team studied this at scale and found that meaningful interactions between concurrent tests were rare in practice. Their write-up A/B Interactions: A Call to Relax is worth reading before you decide to isolate every test.
Read more: Minimum Detectable Effect (MDE) Explained for Ecommerce
When to Use Mutually Exclusive Experiments
Exclusion costs traffic, so save it for the cases where overlap would genuinely muddy the answer. Mutually exclusive experiments make sense when:
- Two tests change the same element. If both tests edit the add to cart button, a shopper in both variants sees a third version that neither test planned.
- The changes deliver the same message. As in the free shipping example, overlapping tests can hide each other’s effect.
- The combination could break something. Conflicting scripts, layout clashes or two discounts that stack in checkout are good reasons to keep tests apart.
- The decision is expensive. For a pricing change or a new checkout flow, you want the cleanest read possible, even if it takes longer.
For everything else, overlap is the better default. Tests on the homepage, a collection page and the checkout can share traffic with very little risk.
A useful middle ground is to borrow the layer idea. Put tests that touch the product page in one exclusive group, tests that touch the cart and checkout in another, and let the groups overlap with each other. Each shopper sees at most one product page test and one checkout test at a time.
Traffic Planning for Concurrent AB Tests
This is where small stores feel the trade-off most. The table below assumes a store with 100,000 visitors a month, a 3% conversion rate, and tests designed to detect a 15% relative lift at 95% confidence and 80% power. That needs about 24,200 visitors per variant, or roughly 48,400 per two-arm test.
| Number of tests | Overlapping (each test gets all traffic) | Mutually exclusive (traffic split between tests) |
| 1 test | About 15 days | About 15 days |
| 2 tests | About 15 days for both | About 29 days each |
| 3 tests | About 15 days for all three | About 44 days each |
| 4 tests | About 15 days for all four | About 58 days each |
With overlap, every test sees the full traffic and finishes in about two weeks. With exclusion, the same four tests take about two months each, which is often longer than a store can wait. The longer a test runs, the more it is exposed to promotions, seasonality and cookie loss. If your traffic is lower still, start with our guide to calculating A/B test sample size before you plan the calendar.
The CustomFit.ai team’s guide to testing velocity and how many tests to run makes the case that more disciplined tests each month means faster learning. Overlap is often the cheapest way to raise that number without buying more traffic.
A Simple Process for Running Several Tests at Once
- 1. Write each hypothesis and primary metric before launch. This keeps the review honest when results come in.
- 2. Group tests by area. Product page, cart and checkout, and site-wide messaging are sensible starting groups.
- 3. Make tests in the same group mutually exclusive. Let different groups overlap.
- 4. QA every combination. Click through the variant pairs that real shoppers can land in, on mobile and desktop.
- 5. Check sample balance daily. A broken split ruins a test faster than any interaction. Our post on sample ratio mismatch shows how to run the check.
- 6. Segment the result by the other live tests before rollout. Ship only when the lift holds across them, or when you understand why it does not.
- 7. Measure the combined effect. After shipping several winners, a holdout group tells you whether the total gain matches the sum of the parts.
Test More Ideas Every Month
Run overlapping tests on your product, cart and checkout pages without writing code. Try CustomFit.ai free for 14 days, no credit card required.
Summary
Running multiple AB tests at the same time is safe in most cases, because random assignment spreads the effect of each test evenly across the arms of the others. The exception is an interaction effect, where two changes amplify or cancel each other. Look for tests that edit the same element, repeat the same message or could clash visually, and keep those in mutually exclusive groups. Let everything else overlap. For a store with modest traffic, overlap can mean finishing four tests in two weeks instead of two months.
FAQs
Is running multiple AB tests at the same time safe?
In most cases, yes. If every test splits visitors at random on its own, shoppers from the other tests land in both arms in roughly equal numbers, so their influence washes out. Be careful with two tests that edit the same element or repeat the same message. That pairing is where an interaction effect tends to show up.
What is an interaction effect in AB testing?
An interaction effect happens when the lift from one change depends on the variant of another test. For example, a free shipping message on the product page and a free shipping bar in the cart may each work alone but add little together, because they repeat the same message.
What are mutually exclusive experiments?
Mutually exclusive experiments are tests set up so that each visitor can join only one of them. No shopper sees both changes together. This removes interaction risk but splits the traffic, so each test takes longer to reach its sample size.
How many concurrent AB tests can a small store run?
With overlapping tests, a small store can run as many as it has sound ideas for, because each test still gets the full traffic. With mutually exclusive tests, each extra test divides the traffic, so a store with 100,000 monthly visitors may wait about two months per test when running four at once.
Do overlapping experiments invalidate test results?
Usually not. Overlapping experiments give a fair estimate of each change as the site runs today, with the other tests live. If you suspect an interaction, segment one test’s results by the variant of the other. A large, consistent gap between segments is the signal to look closer.



