A/B Testing (Split Test): Meaning, Process and Examples

An A/B test is an experiment in which two variants, such as of an ad, an email or a page, are shown at random to comparable groups in order to measure which works better. It is also called a split test.
Also: split test, split-test, A/B testing, controlled experiment
How A/B testing works
- Set a goal, such as the conversion rate or the click-through rate
- formulate a hypothesis and change exactly one thing
- distribute visitors randomly between variant A and B
- let it run until there is enough data for a reliable conclusion
- evaluate the result and adopt the better variant
What you can test
- Headlines and ad copy
- Images and ad creatives
- Email subject lines
- Buttons, forms and page layout
- Offers and prices
Common mistakes
Stopping too early, changing too many things at once and using samples that are too small are the most common mistakes. Seasonal fluctuations can also distort a result. The standard work on online experiments is the book by Kohavi, Tang and Xu.1
A/B testing in advertising
In ad campaigns, the split test is essential: in the end, the creative that sells is the one that runs, not the one people like. Platforms such as Meta and Google Ads offer their own testing features.
Frequently asked questions about A/B testing
What is meant by A/B testing?
Comparing two variants under the same conditions, with users randomly assigned to the variants. This lets you measure which variant achieves a goal better.
What are some examples of A/B testing?
Two subject lines of a newsletter, two headlines of a landing page, two ad creatives or two colours of a booking button.
How long should an A/B test run?
Until there is enough data for a statistically reliable conclusion, and for at least full weeks so that fluctuations between days of the week even out.
Related terms
- ConversionMarketing & Advertising
- CTR (click-through rate)Marketing & Advertising
- CopywritingMarketing & Advertising
- Performance marketingMarketing & Advertising
Sources
- Kohavi, R., Tang, D., Xu, Y. (2020): Trustworthy Online Controlled Experiments. A Practical Guide to A/B Testing. Cambridge University Press.
← All terms in the marketing glossary
Simon Gütter is a marketing expert and has worked for years at the intersection of marketing, sales and automation. He finds the places where companies lose time and customers, and builds processes that run on their own: from appointment booking to reporting. More about Simon