Introduction to A/B Testing

An explanation of A/B testing for event creators.

Introduction to A/B Testing

A video on Facebook Ad campaign structure and A/B testing and why they're important.

A/B testing, or split testing, is one of the most important practices for any marketer. To understand what A/B testing is and why it's so important, we'll review Facebook ad campaign structure.

Every Facebook ad campaign contains three elements:

• The first is the overall stats of the campaign. Whenever you press ‘Boost’ on a post, there’s a campaign in Ads Manager with a title showing your overall spend, financial metrics, and engagement metrics.

• The second element is the ad set of the campaign—in other words, the campaign’s audience. Who’s being served the ads in this campaign? Where do you want your ad located? Which individuals do you want to target?

• Finally, there’s the ad creative for the campaign: the post’s look on Facebook, Instagram, and other placements.

Within a Facebook campaign, you can have multiple ads with different creative and distinct audiences. Surprisingly, though, that’s not the kind of campaign we come across most often. Too often, we see one ad set with interest-based targeting and one ad creative, which means the campaign isn’t taking advantage of A/B testing.

What are the drawbacks of creating a campaign without A/B testing?

To the right of each of these boxes, you’ll see spend, revenue, and return on ad spend. For this hypothetical ad campaign, $100 was spent and $50 was made in revenue. Therefore, the return on ad spend is about 0.5. For every dollar spent, you made $0.50 back in gross ticket sales.

This is what campaigns tend to look like for most event organizers. They represent a missed opportunity—and a poor gamble. Without A/B testing, you might spend the entire marketing budget for your show on a single ad set with a single creative—and not necessarily hit the right fans.

That’s why we recommend A/B testing, or split testing: It’s the best way to improve your overall return on ad spend on your campaigns.

What is A/B testing? Think of it as running a controlled experiment. Instead of advertising to one ad set with one creative, you include multiple ad sets and multiple creative at the outset of your campaign to see what performs better. Once you know which performs better, you can allocate more budget to the higher-performing ad sets and creative combinations, boosting your return on ad spend.

Here’s what A/B testing looks like at the structural level, with another hypothetical ad campaign. We’ll use the same $100 we spent in this previous example, but this time we’ll split that budget evenly over two different ad sets.

A/B Testing at the Structural Level

Here’s the first ad set from our initial example to the left. $50 was spent; $50 in revenue was made, so the return on ad spend is 1x. Two creatives were made: ad one and ad two. Of those two, the second ended up yielding a higher return on ad spend.

This is the second ad set to the right. Here, we split the budget and put $50 toward a second ad set (in this case, mailing list combined with website traffic). This ad set outperformed the first. It yielded a higher return on ad spend: about 5X. Plus, that 5X came from directing budget toward two different ad creatives. Spending $25 on ad one made you $100; spending $25 on ad two made you $150.

By including multiple ad sets, we discovered which audience came with more attractive customer acquisition costs—which, in turn, resulted in more sales. By including multiple ad sets, each targeting a slightly different group of fans (i.e., a custom audience or another set of interests), you may actually find a group of fans that is more willing to buy tickets at a more favorable cost. Your $100 will net you $300 rather than $50.

The best part about A/B testing is that you don’t need to limit yourself to two ad sets.

Here’s an example of a campaign where we used six ad sets. Notice how the performance of these ad sets differ? The two ad sets on the bottom have very high return on ad spends. The two ad sets on the top have low return on ad spends.

If we had only included one audience in this campaign (i.e., the interest based audience), we might not have sold any tickets whatsoever. The $1,500 we spent might not have resulted in any sales at all.

This is why A/B testing is crucial to any event marketer’s gameplan. By including more ad sets in your campaign, you’ll find different segments of fans interested in buying tickets and see a much higher return on ad spend as a result.

Next up is budget optimization:

Introduction to Budget Optimization