What is A/B Testing?

A/B testing means experimenting with two versions of a website, app, or other marketing property to determine which version best achieves the goals and proves the hypotheses of the experiment.

Marketers often refer to A/B testing as split testing. The split refers to the fact that there are two versions of a marketing asset (an A version and a B version). Throughout the data collection process, visitors or potential clients are served either the A version or the B version, and based on the user’s actions, marketers determine which version fulfills their hypothesis.

For example, a company could create a variation of their current website homepage and A/B test the existing version with the variation. They would serve the variation to a certain percentage of visitors and compare the performance of the variation to the performance of the existing homepage.

The performance comparison would be based on the goals for the site. Did more users click to the pricing page from one version? Or did more users sign up for newsletters from the other? Depending on those goals, one variation might perform better and can then be implemented for all visitors.

Why Should Marketers A/B Test?

A/B testing is a safer way to make changes to marketing materials and determine how those who interact with the materials will react. By testing changes on a subset of people, you ensure that any adverse reactions only happen on that subset. Without A/B testing, any drops in traffic or conversions because of the changes could affect 100% of the people who come in contact with that marketing asset. That’s a huge potential for loss.

A/B testing also allows marketers to potentially make decisions with less data than is statistically significant. Sometimes it’s just obvious when an A/B test is failing or succeeding. As long as the data makes sense (for instance, it’s not a holiday for your market or something like that), A/B testing can tell you if you hypothesis is correct pretty quickly.

How Do Marketers Perform A/B Tests?

For digital assets like websites and apps, there is software (like Experiments in Google Analytics, Optimizely, or VWO) that lets marketers show a certain asset to a set number of visitors. For marketing pieces like email, marketers can also use testing software that’s a part of their email service provider or try to segment their lists and send the A and B variations manually.

For print pieces, A/B testing can vary depending on the asset and how it’s seen by the target audience–business cards, direct mail pieces, print ads, etc. The idea is that you’d create two versions of each with a single difference between the two, and test which achieves your goals better.

Can I A/B Test More than One Thing?

In general, this can muddle your data. If you want to test three separate subject lines for an email, that makes sense to test A/B/C against each other at the same time. However, testing a separate subject line, a call to action and email design means that when you get the data, you don’t know which aspect truly drove the difference in conversions.

It’s better to test one element of a piece at a time versus multiple (called multivariate testing) for clean, accurate data.

What is Considered Statistically Significant Data?

This sounds a lot fancier than it has to be. Statistical significance just means it makes sense that the relationship in your data is caused by more than just chance. That is, you have enough data and there’s enough of a correlation to determine that the change in your email subject line WAS the actual reason it got more opens than the other.

There is definitely a more technical definition, but for A/B testing if you use a software like those mentioned above, they typically do the major math for you. A/B Test Guide actually has a good calculator to help you determine if your A/B test results mean what you actually think they mean.

So, all that’s left is to get testing your data! For marketers, one potential A/B test idea includes listening to call recordings to get specific verbiage that your clients are using, and testing that on your site!

To learn more about other ways call recording can improve your business, request your no obligation demo of CallRail. Or go ahead and start your 14 Day Free Trial, no credit card needed.