Subject Line A/B Split Testing

A/B split testing is a great practice for all types of marketing, but what is it? How does it apply to email marketing? I’ve got you covered!

A/B split testing simply entails taking a sample set of your overall database and sending two versions of your email — one to each half of that sample set. I always like to adhere to the 80/20 rule: 20 percent of the entire database becomes the sample set. Using this metric:

10 percent of those on my list get “email A.”
10 percent get “email B.”
The remaining 80 percent receive the winning version from the split test.

These numbers are not cast in stone, and there’s room to make your own judgment. For example, if working with a really small list, of say 200 people, I’ll refine these metrics. In this example, I’d likely do 60 percent/40 percent. So, 40 people receive “A,” 40 receive “B,” and the remaining 120 receive the winner. Otherwise, without this modification, the 80/20 rule would result in such small split groups, what’s the point?

The purpose of the A/B split test is to allow data to reveal which of your variables resonates better with the audience. Most email service providers (ESPs) offer functionality to do a split test on subject line, design or content. I always, always split test the subject line only. Why, you ask? Well, simply put, the strength of your subject line is the key indicator of your open rate. If no one opens your email, who cares what your design and content look like? (Sorry designers, writers and developers!) Because of this rationale, I always use subject line split testing as my secret weapon to maximize the open rate.

In fact, using this secret weapon left jaws on the floor for a campaign we ran a couple of years ago. By doing an A/B split test for a client, we were able to increase the open rate by 69 percent! Yes, you definitely read that right — that garnered us an extra 3,131 opens, increasing our click rate enormously (from zero clicks to 82). The funny thing is that our subject lines weren’t even wildly creative. Here’s what we used:

A (winner): On the House
B (not “winning”): Take Advantage of our October Promo

As you can tell, by taking two minutes to set up a split test (it was an hour waiting for the results), we killed it with generating leads for our client. Not to brag, but merely to add more credibility for the doubters, because of the success of this campaign, Campaign Monitor hand-selected us to be in its “Top 100 Email Marketing Campaigns of 2014” resource guide — selected from more than 300 million emails sent that year.

Now that you’re basically sold on this idea, you’re likely wondering how to get in on the action. Rest assured, it is really simple. A solid ESP (such as Campaign Monitor, for example) will make this really easy for you. All you need to do is use the interface to simply select the type of split test you’d like to run and input your two variables (input type depends on what you’re testing — subject line, design, content, etc.). Then simply select the amount of your list to become the sample group, set the duration you want to run the split test for (I usually go for an hour), and fire away. Sit back and watch the metrics do the heavy lifting.

Since you’re now equipped with the secret weaponry, good luck and enjoy the extra leads!