E-Mail Metrics Are Broken. Here’s How To Make The Best Of It.

In the radio industry, we are ((cough)) perhaps more accustomed than most to making business decisions based on numbers where the methodology for collecting and reporting them are, shall we say, not transparent.

That’s more and more common in the digital space as well where email opens are getting less and less reliable across the board.

Most marketers don’t have an in-house technology resource and rely on SaaS vendors such as the MailChimps and Constant Contacts of the world. As such, they have limited visibility into what’s going into these metrics and even less control over how to fix them.  Even diagnosing the problem can be tough as there are sometimes only the vendor’s numbers to go by.

A few weeks ago we began to seriously question the data that our email provider was giving us. We would see hundreds of opens and clicks in the email reports, and then just a tiny trickle on the landing pages. Something was janky.

After a bit of testing on our own email campaigns and selected clients, we’ve come to believe that for certain vendors the KPI’s we’re used to relying on (Open Rate and Clicks) are inflated by 30-60%. In some cases and with some lists, it can be as high as 90-95%.

The way we arrived at that conclusion was by adding a “hidden” link in the code that only a bot or script would be able to find and click and subtracting that number of clicks from the total.  This isn’t a perfect method by any means and thanks to increased privacy by the tech providers we’re also often not able to see or properly attribute actual clicks, so we adjusted for that as well.

Where are these bots and scripts coming from you might add? Corporate firewalls, anti-spam measures, and anti-virus applications scanning the links for malware and phishing.

Tracking pixels are filtered out by privacy enhanced mail clients when they hit a user’s inbox, but triggered by an automated malware scanner. That same scanner will then dutifully click every single link in the email to make sure it doesn’t lead to a malicious destination, and that’s what shows up in the report.

Meanwhile, tools like Google Analytics or Adobe Analytics are set to filter out clicks it thinks are from bots, so they aren’t registering at the destination.

So what’s the solution?  There isn’t a perfect one. We can only attempt to read the tea leaves here based on the limited hard info we’re getting.

The first thing to do is to consider your email marketing reports as directional indicators vs a literal number of opens and clicks. Honestly, the most useful information you can probably glean from those reports is your overall deliverability. There’s not a reliable way to discern whether a click or open is legitimate or not, but if it’s happening you can at least rest assured the message got delivered.

In terms of measuring effectiveness there are a few things you can do.

1. Pay more attention to your conversions, wherever they take place.  On a commerce site that’s top-of-funnel activity, on a lead generation site it’s signups, and on a media site it can be simply be the raw pageview.
2. Set up unique URL parameters for your email campaign, so you can attribute the activity to email. This page would go from just https://nuvoodoo.com/2023/02/27/e-mail-metrics-a…e-the-best-of-it/ to https://nuvoodoo.com/2023/02/27/e-mail-metrics-a…e-the-best-of-it/?src=email.
3. Use unique landing pages for your email campaigns that can’t be found any other way.  That way you know any traffic it does get comes from email.

A lot of CRM and email marketing apps like to sell you on how great their analytics and user tracking is.  Sadly, until those providers put a lot more effort into filtering out the noise the safest thing to do is regard that information as fiction and focus on the real life metrics you can see.