It’s the holiday season! For some, this is the time of year for family, friends and reflection. For the other 97 percent* of us, it’s time to shop! America’s obsession with Black Friday, Cyber Monday and the rest of the holiday shopping season has never been stronger. Or weaker? Or something? All I know is that you should be skeptical of anything you see regarding the Thanksgiving weekend performance.

And now, I will tell you about the Thanksgiving weekend performance

We’re not discussing revenue in this post. Instead, we’ll dive into the weekend’s email subject lines – more specifically, how “percent off” deals affected email open rates.

As everyone knows, Black Friday and Cyber Monday are the days for deals. Juicy “percent off” offers motivate customers to buy, buy, buy. But is the conventional wisdom, that “a deep discount will get people to engage with my brand,” actually right?

A few weeks ago, my counterpart in the UK published an analysis of how percentage off discounts influence open rates. Taking the cue from Karl, I wanted to expand this analysis into the U.S. market, paying special attention to Thanksgiving weekend. To begin, I gathered data on a few thousand mailings from our largest retail clients. To determine the baseline expected open rates, I averaged each brand’s performance in the 6 weeks prior to Black Friday. I then analyzed all the mailings sent on Black Friday and Cyber Monday, dividing the subject lines based on the appearance of a percentage off offer. Interestingly, percentage off offers were less prominent than I expected:

And when percentages off were present… their values were all over the place:

Higher volume doesn’t lead to improved performance

Conventional wisdom would suggest that advertising a discount more frequently would lead to better performing discounts. The data, however, doesn’t support that idea. When I looked at volume distribution and relative performance for each advertised discount, I found a relatively strong negative correlation of -0.63. So the more frequently a discount was advertised, the worse it tended to perform. We can see this visually in the chart below:

On average, advertising discounts did not significantly improve open rates. What happened?

The first thing to note here is the wide spread in the data – some percentage off discounts worked very well! Overall, though, shouting about a discount wasn’t what convinced customers to open emails during the holidays.

But maybe it wasn’t just the percentage off discounts that faltered this season – perhaps all opens were down?

As you can see in the histogram above – this wasn’t the case. The average mailing not touting a percentage off discount did ever so slightly better than the baseline average. Still, the spread of data is very wide, with a lot of variation in results.

It could be that the dispersion of results was a product of each brand’s initial baseline; brands that normally had great engagement would see positive gains for percentage off discounts while brands with poor engagement would see little to negative lifts, or vice versa. But this hypothesis was also proven incorrect, as the relative starting place for each brand versus the discount performances had a correlation approaching zero. No matter which way I sliced it, the performance of discounted subject lines were more or less random.

Ultimately, this last point is the most important. The subject line, for all its ubiquity and focus, is probably a lot less influential than we tend to believe. Sure, a subject line can be optimized, carefully crafted to invoke the greatest lift in response possible, but the baseline expected performance is influenced by a much larger conversation – the one between the brand and its customers. If the brand relationship has been cultivated and refined through intelligent interactions and sophisticated targeting, the open rate is likely going to be higher. If every marketing message simply shouts, DISCOUNT, DISCOUNT, DISCOUNT, and there is no larger value-add, engagement probably won’t be great. Advertising a discount in a subject line might really help get people involved – or it might not.

So what is the future of the subject line? Are they worth the disproportionate time and energy that marketing organizations tend to spend on them? Or should we recognize that their importance is probably minimal? The truth is, it’s a little bit of both. Subject lines are important – they are the first impression and often the first interaction of the day with a customer. But their importance is likely inversely related to the strength of the brand (the “from” line, if you will). The stronger the relationship is, the less important the subject line becomes. Maybe that’s the ideal – a perfect “from” name, one that tells you more about what’s inside the message than a subject line ever could.

*Not a real stat

Connect with Jacob Davis, Senior Analyst, on Twitter: @davisj2007.