Many-Armed Bandits in UX Experiments: Faster Testing with Smarter Traffic Distribution

Traditional A/B testing splits incoming traffic equally (50/50%) between design versions A and B. Then, half of the incoming traffic will receive the original version and the other half will receive the new design variation during the experimentation period. This fixed, equal distribution of traffic generates statistically more accurate results with unbiased data, but it is slow and negatively affects the conversion rate during experimentation because users who receive the low-performing design version may not convert.

The multi-arm bandit strategy allows you to perform A/B, A/B/n, or multivariate testing faster by dynamically distributing traffic based on the winning version. Let’s understand how multi-armed bandits manage A/B testing faster without affecting the conversion rate by comparing with the traditional traffic distribution method of A/B testing.

What are multi-armed bandits?

In UI/UX A/B testing, the multi-armed bandit (MAB) strategy is a dynamic incoming traffic distribution method that redirects more traffic to the current winning design version. It initially starts with a traditional 50/50% split, but then dynamically increases traffic to the highest performing version to stabilize the test metrics and selected traffic percentages to conclude the test faster:

MAB is not just for A/B testing: you can use it to optimize traffic for A/B/n and multivariate testing.

The slot machine example is the simplest way to understand MAB:

Suppose there are two slot machines with unknown reward rates. How to find the most rewarding machine without losing more money?

• First, you play equally with A and B
• If machine B works well, you will use it more often
• Still uses machine A from time to time and starts playing it often if it rewards better
• Select the machine that rewards the most

How do multi-armed bandits work in UX?

Let’s discuss each generic step that a MAB A/B test performs to optimize traffic:

1. Fair initialization: Initializes the A/B test with a 50/50% traffic split to give each version a good amount of traffic. In A/B/n testing, the algorithm initially distributes traffic based on the number of versions, for example 25% for a four-version A/B/n test.
2. Monitoring: Monitors changes in metrics, identifies the best performing version for A/B testing, and ranks all versions for A/B/n and multivariate testing.
3. Traffic distribution adjustments: Adjusts traffic distribution percentages based on monitoring results. For example, if version B has 5% conversion, but version A only has 4%, the algorithm increases traffic for version B because it is more successful.
4. Stabilization: The goal of the algorithm is to stabilize the metrics to stop the process. This typically occurs when new traffic does not greatly change the current traffic distribution percentages.
5. Stop: The MAB process will stop when the traffic distribution is stabilized (the dominant version can stabilize the traffic percentage between 80% and 95%) and no significant changes in the metrics are observed. For example, if the conversion rates for versions A and B stabilize at 6% and 5%, the process will stop

Benefits of using multi-armed bandits

Using MAB instead of traditional fixed traffic distribution has the following advantages:

• Shorter tests: MAB testing stops when a specific version dominates. The test is therefore generally faster than a traditional A/B test which waits for the result to reach statistical significance.
• Best conversion during experience: The better performing version generates more traffic, so the conversion rate will increase, even if a specific version performs poorly, unlike the fixed and equal distribution of traffic.
• Less wasted traffic: MAB optimizes inbound traffic during the experiment by sending less traffic to poorly performing versions, so inbound traffic is not wasted.
• No fixed initial sample size requirements: No need to calculate and meet statistical significance requirements for incoming traffic before starting the test: flexible start and progress depending on traffic flow

Limits and pitfalls of multi-armed bandits

MAB has advantages, but the following problems have caused designers to rethink its use compared to the classical statistical method:

• Depends on the MAB algorithm: The accuracy and reliability of the result depends on the quality of implementation of the MAB algorithm that dynamically divides the traffic. Weak algorithms can generate inaccurate results
• No exact stopping time: You don’t know exactly when a MAB test will end because it waits for the traffic distribution to stabilize, unlike traditional A/B tests which start with a predefined duration.
• Statistical bias: MAB starts with a fair distribution of traffic, but dynamically adjusts the percentages later, so the result is skewed compared to the initial traffic.
• Statistical significance is not guaranteed: You cannot present MAB test results with the usual 95% statistical significance because it does not use an assumption-based statistical foundation for decision making.

Many-Armed Bandits vs. Traditional A/B Testing

Here is the summary of MAB versus traditional A/B testing comparison:

Example use cases

A MAB test works best when you need to optimize traffic while quickly evaluating design versions. Here are some examples:

• CTA: CTAs decide the product conversion rate, so MAB helps in testing design versions without losing traffic
• Onboarding flow: Testing two integration flows with a fixed 50/50% traffic split can increase the product abandonment rate if one version performs poorly. Using MAB prioritizes the best performing stream without running the experiment for a long time
• Recommendation systems: Imagine you need to perform a comprehensive test for two e-commerce product recommendation algorithms on a homepage. Using MAB helps you find the right algorithm, also maximizing revenue, unlike traditional A/B testing

Tips for designers

Here are some practical tips for performing effective MAB testing:

• Avoid premature conclusions: Do not stop the test forcefully even if you see a rapid increase at the beginning of the phase, always wait for the traffic and metrics to stabilize.
• Continuously monitor: Monitor top-performing version changes, seasonal traffic effects, and shut down properly without overrun
• Combine with human knowledge: The results depend on the quality of the multi-banding algorithm and the initial traffic, so analyze the results yourself without blindly trusting the algorithm’s output.

Conclusion

The multi-armed bandit strategy may be preferred over traditional traffic distribution with A/B, A/B/n, or multivariate testing if you care more about wasted traffic and don’t care much about statistically significant results.

FAQs

Is the many-armed bandit method better than traditional A/B testing?

It depends on the scenario. MAB is better if traffic loss is critical, and traditional A/B testing is better when you prioritize statistical significance

Should I calculate traffic percentages frequently and notify developers?

No, A/B testing tools that support MAB handle everything: you just need to initialize the test