How much difference can a email subject line really make?
Who is this article for?
This blog post is written for education suppliers who send email campaigns to schools and want to get more from their subject lines, using real A/B test data rather than general rules of thumb.
We spend a lot of time looking at email campaign data at Buzz. And one of the things we’ve been looking at recently is A/B testing. More specifically, subject lines.
It’s one of those areas where everyone has an opinion. Keep it short. Ask a question. Put the offer in the subject line. Don’t put the offer in the subject line. Create some curiosity. Be really specific.
The problem is, two perfectly sensible subject lines can produce very different results. So rather than trying to decide which one sounds best, we’ve been testing them. And some of the differences have been much bigger than we expected.
One subject line. 138% more clicks.
That was probably the result that first caught our attention. The two emails were promoting exactly the same thing to the same type of audience. The difference was the subject line.
The better-performing version generated 44% more opens. Good result. But more importantly, it generated 138% more clicks. That’s a pretty significant difference from changing a few words at the top of an email.
And it wasn’t the only example.
31% more clicks than the other
In another campaign, changing the subject line barely affected the open rate at all. But one version still generated 31% more clicks than the other. We actually think that’s the more interesting result.
Opens are useful, but they’re not usually what our clients ultimately want. They want teachers and school decision makers to read the message, visit the website, download something, enquire or take whatever the next step happens to be. A subject line isn’t just about getting an email opened. It can change who opens it and what they do afterwards.
The stat that really stood out to us
When we looked across the results, 75% of the tests showed an improvement of 20% or more in at least one key engagement measure. That’s enough of a difference to matter.
Imagine two versions of a campaign going to the same audience. Same product. Same client. Same underlying message. One performs 20%, 30% or even 100%+ better simply because of the way the email was introduced. If you’re already investing in getting your message in front of thousands of teachers, that’s worth knowing.
So, what makes a good subject line?
This is where we’d love to give you a clever formula. Unfortunately, our data doesn’t really support one.
We’ve seen direct subject lines work. We’ve seen questions work. We’ve seen benefit-led messaging beat product-led messaging. We’ve also seen relatively small changes produce surprisingly different results. And that’s really the point.
“There isn’t one type of subject line that teachers always respond to. It depends on the audience, the product, the proposition and probably a fair few other things besides. That’s why we prefer testing to guessing.”
For one campaign we might test a direct benefit against a question:
Free resources to support your teachers this term
Could this make life easier for your teachers?
For another, we might test putting the offer front and centre against leading with the problem the product solves. The important bit isn’t necessarily which approach wins. It’s finding out.
The bit that’s easy to overlook
There’s another reason we’ve become increasingly interested in A/B testing. The learning doesn’t disappear when the campaign ends.
Say a client runs several campaigns with us during the academic year. If we learn from the first campaign that their audience responds far better to a direct, benefit-led subject line, we can take that into the next one. Then we can test something else.
Over time, we’re not just trying to improve individual email campaigns. We’re learning more about what actually gets that client’s audience to engage. That’s considerably more useful than following a list of generic email marketing rules.
It’s the same reason we spend so much time analysing our wider campaign data. Our Email Marketing Top Trends research looks at what we’ve learned from campaigns across the education sector, while our analysis of the best time to email teachers tackles another question we’re regularly asked by clients.
A/B testing takes that thinking one step further. Instead of asking what generally works when marketing to schools, we can start asking: what works for you? And that’s a much better question.
If you’re planning an email campaign to schools, you can find out more about our email marketing service or talk to the Buzz Education team.
A few quick questions about subject-line testing
What is subject-line A/B testing?+
It’s simply a way of comparing two subject lines. Different versions are tested with comparable audiences so you can see which generates the stronger engagement.
Does changing a subject line really make that much difference?+
Sometimes, yes. We’ve seen subject-line changes produce substantial differences in both opens and clicks, including one example where the stronger version generated 138% more clicks.
What should you test?+
There’s no single formula. You might compare a question with a statement, a short subject line with something more descriptive, or lead with a benefit in one version and an offer in another.
The useful part is finding out which approach your own audience responds to.
Is A/B testing only about improving open rates?+
No. In fact, we’re particularly interested in what happens to clicks. We’ve seen tests where open rates were very similar but click rates were noticeably different, which tells us something about the type of person each subject line encouraged to engage.
Popular articles
