The Basics of Email Metrics: Are Your Campaigns Working?

You already know the value a good email campaign can bring to your nonprofit, whether it is an eNewsletter, or email advocacy and fundraising appeals. But how can you tell if your electronic communications are actually having the impact you hoped for? Is the work you’re putting in worth the effort? Email metrics can help you determine the effectiveness of your communications and fine tune them to improve their efficiency.

In this article, we’ll explain how to use basic email metrics to understand how your constituents are reacting to your emails. We’ll also talk through the data points needed to track email performance, the most common formulas for standard email metrics, what they can tell you, and how to use them to improve results.

What Data to Collect

• Messages Sent. This is the number of outbound emails sent as part of a particular mailing.
• Messages Delivered. This is the number of sent emails actually delivered to recipients’ inboxes. If they’re not delivered, that means they’ve “bounced”.
• Hard and Soft Bounces. A hard bounce is a permanently undeliverable email—for example, one sent to an invalid email address (joesmith@yahoo.con) or to an address that no longer exists. A soft bounce is an email that is only temporarily undeliverable—for example, to a recipient whose mailbox is full. Ideally, you should track both.
• Unsubscribes. This is the number of individuals who unsubscribe from your list in response to each mailing sent.
• Messages Opened. This is the number of recipients who open your email to read it. Due to the way open rates are tracked and the rise of image-blocking software, this number will never be accurate, but can still be useful.
• Click-Throughs. This is the number of times any recipient clicks on any trackable link within the email. Ideally, each link should be counted only once, even if it is clicked on multiple times.

How to Capture This Data

Data about your emails will come primarily from the bulk email software package that you’re using. Many of these packages - such as ConstantContact [2] or VerticalResponse [3] - provide reports with key data points for each of your mailings. Check your existing package to see what it can do.

Most email packages will track messages sent, delivered and opened, and click-throughs. Many will provide the number of unsubscribes, as well. But the number of hard and soft bounces may be a little more difficult to find in an email package. And you’ll likely need to track responses on your own, unless you’re using a tool that integrates email functionality with the ability to support the actions you're requesting—such as making a donation or emailing a congressman. When you send out an email that asks your constituents to take an action, make sure to take note of how many people perform that action over the next few days.

If your software doesn’t let you track data points, or if you’re using something that wasn’t designed for bulk email (like Outlook), your options are limited. Short of building your own tracking mechanisms—which would be an expensive exercise in reinventing the wheel—you’re reliant on the numbers your emailing software package provides for you.

If you're serious about tracking and improving your email campaigns, it's worth investing in a software tool that can help in your quest to track metrics. A number of software packages provide solid services and metrics without a huge investment. For instance, Vertical Response [3] offers solid functionality and reports free for up to 10,000 emails a month for 501(c)(3) nonprofits. Network for Good's Email Now [4] is also solid and well priced.

Moving from Numbers to Information

Even the simple data points mentioned above can be useful as standalone data. For instance, an increase in number of click-throughs from one month to the next means more people have seen your message, which is always a good thing. Conversely, a significant discrepancy between the number of emails sent (messages sent) and the number delivered (messages delivered) can be an indicator of trouble. Why aren’t emails reaching readers’ inboxes? Are you sending to old email addresses, or gathering addresses in a way that encourages fake addresses? Or maybe a particular ISP (like Hotmail or AOL) is blocking your organization's email messages, or routing them directly to junk mail folders (unfortunately, this can and does happen.)

For most organizations, though, the real power comes from calculating simple metrics and comparing them across email campaigns to see what tone, timing and topics are most effective.

This is where software packages can get confusing. Many vendors provide some built-in metrics, but the equations used to calculate them aren’t standardized. Different software packages calculate them in different ways, and it’s not always easy to figure out how.

It can be helpful to use an Excel spreadsheet to track the basic numbers provided by the email software, and then calculate your own metrics. That’s also a great way to understand what really goes into these statistics.

Making Sense of The Data: The Metrics

What are the useful metrics, and what can they tell you? Let’s take a look at a couple of particularly helpful calculations.

Open Rates

Open Rate determines what percentage of recipients actually open your emails. To calculate Open Rate, divide the number of messages opened by the number of messages delivered:

Open Rate = # of messages opened / # of messages delivered

The Open Rate measures recipients’ interest in the From and Subject lines of your email. These two fields have just one job—convince recipients to open your email. After that, the text, layout and any images in the email can take over.

Good Open Rates range between 15 percent and 25 percent. However, open rates are declining across all organizations as a result of changes in how email clients are handling images. Because software packages can’t know whether someone actually opened an email, they calculate the Open Rate based on whether the recipient views a tiny image file hidden in the body of the email. Because more and more email clients default to blocking all images, or showing only plain text, people can open the email without being counted by the software package.

This means that the Open Rate is a questionable metric at best. It can still be a useful way to compare the success of one subject line to another, but shouldn't be considered a measure of who's actually seen your email.

To Improve your Open Rate

• Create more compelling From and Subject lines. This seems commonsense, but it’s not always clear what it actually means in practice. Think about what subject lines make you open an email in your own inbox—good rules of thumb include asking a question, keeping subject lines under six words and using lowercase in all but the first word and proper nouns.
• Test your subject lines. There are several ways to do this. If you have more than 10,000 or so people on your list, create two separate emails with the same body information but different subject lines, and send each to half your list. Measure the Open Rate for both emails, and see which one did better. As you do this a number of times, you’ll start to develop a better sense of what works well for your particular audience. Or, if you have a very large list—hundreds of thousands of people or more—you can even test two subject lines on a smaller percentage of your list, pick the one with the higher open rate and send it to the remainder of your list. (This type of testing is called A/B testing, as you’re measuring the performance of email A against email B.)

Click-Through Rates

A second key metric, Click-Through Rate (CTR) is the number of click-throughs divided by the number of messages delivered.

Click-Through Rate (CTR) = # of click-throughs / # of messages delivered

The CTR tells you both how interested recipients were in a particular issue, and how well you converted that interest into click-throughs. Advocacy appeals typically have a much higher CTR than fundraising appeals. If your organization sends out both types of emails, track the results separately to predict results from both types of communications.

In general, expect to see CTRs between 1 percent and 10 percent. If you're asking people to take an action that doesn't involve money—for instance, emailing their congressman, taking a survey or getting a free bumper sticker—you're likely to see a much higher Click-Through Rate than you would with a fundraising email. eNewsletters fall somewhere in the middle.

There are a number of variations on CTRs. One common variation, which we’ll call an Adjusted Click-Through Rate (ACTR), is calculated by dividing the number of click-throughs by the number of messages opened:

Adjusted Click Through Rate = # of click-throughs / # of messages opened

While a basic CTR will be lowered when few people open an email, the Adjusted CTR removes the impact of the Open Rate to measure how effectively your email text and format is directing people to your links. Generally, expect to see an ACTR of 10 percent to 40 percent.

Some other CTR variations divide click-throughs by messages sent rather than messages delivered, or divide unique clicks—the number of click-throughs by different users - by messages delivered. Whichever variation you track, make sure everyone in your organization uses the same equation. When comparing your metrics to CTR benchmarks or for other organizations, make sure you know what calculation is being used—the variations can make comparisons confusing.

To Improve your Click-Through Rate

• Make content and descriptions compelling. Provide recipients something they really want to click to see, and a description that makes it clear why they should.
• Ensure the subject line prepares the reader for the content. Generally, readers are more likely to click through when their experience is consistent. If the subject line matches the email content, they'll see what they expect when they open the email, and be more likely to take action.
• Change your format. The format of an email, particularly an eNewsletter, can have a dramatic effect on your CTR. The Adjusted CTR can be particularly useful in accurately homing in on what’s working and what’s not.
• Highlight what you want readers to click on. A call out box—generally an outlined or colored box with a link and a simple call to action—can increase Click-Through Rates, as can buttons and relevant images. Also consider a hotspot—a hyperlinked text-based call to action located at the very top of your email, above even the header or any other html. This ensures that readers will see your message even if they have image-viewing disabled in their email browsers.

Response or Conversion Rate

Another key metric to understand is the Response Rate. Response Rate is the number of desired responses, such as donations, divided by the number of messages delivered:

Response Rate = # of actions taken / # of messages delivered

The Response Rate is the best way to measure the effectiveness of an email campaign. After all, it likely doesn't matter if readers open or click your email if they don't ultimately do what you were hoping they would do.

The Response Rate is also variously referred to as the Conversion Rate, Action Rate, Donation Rate or Advocacy Rate. In every case, the number of desired actions is divided by messages delivered. The key difference in these terms is the type of action being measured. A Conversion Rate typically refers to donations or email sign-ups, while Action and Advocacy Rates generally refer to something like petition signatures or emails sent to Congress.

The rate itself will depend a great deal on what action you're requesting. A good fundraising email Response Rate might range from 0.75 percent to 2 percent. A Response Rate for an advocacy email is likely to be considerably higher than one for a fundraising email, at 3 percent to 15 percent. It can be harder to interpret a Response Rate for this type of email, as people are more likely to pass on the email to their friends. In this case, people who didn’t receive your original email may take action, skewing your numbers somewhat.

It can also be useful to compare the number of actions taken by the number of click-throughs—we'll call this an Adjusted Response Rate:

Adjusted Response Rate = # of actions taken / # of click-throughs

This rate provides more insight if you're trying to optimize your Web site landing page—the page where people actually make the donation or take the action. The Adjusted Response Rate takes out the open and click-through rates as variables to let you see how many people decide not to take action after they click through.

To Improve your Response Rate:

• Provide a compelling reason to act. They way you frame your story and your request has a huge impact on the response rate
• Optimize your landing page. For instance, add a compelling image, remove the distraction of navigation bars and minimize the number of links on the page. This can substantially improve your response rate. Make the action simple and obvious by limiting users' choices as much as possible.

Unsubscribe Rate

The last critical item to track is the Unsubscribe Rate, or the number of unsubscribes divided by the number of messages delivered.

Unsubscribe Rate = unsubscribes/ # of messages delivered

Use the Unsubscribe Rate to measure how well you hold your subscribers’ interest over the long run. If subscribers do not like what you’re saying, don’t find it interesting or feel that you're sending them far too many emails, they’ll tell you by choosing to leave your email list.

If you are acquiring lists of names from outside sources, the Unsubscribe Rate can be particularly important to understand the return on investment. A new list with a very low Unsubscribe Rate is much more useful than one where everyone immediately unsubscribes.

In general, your Unsubscribe Rate should be very low—between 0.3 percent and 1 percent is normal for a single email. However, this number varies significantly by organization and issue, and over time. For instance, expect a temporary increase in the Unsubscribe Rate after acquiring a batch of new email addresses. Unsubscribe Rates may also fluctuate dramatically if your list is small.

To Reduce your Unsubscribe Rate:

• Provide more value. Are you sending emails that are viewed as useful by those that receive them rather than focused solely on your own needs?
• Reduce your volume of email. If you're sending emails more than every couple of weeks and seeing a high Unsubscribe Rate, you might want to consider decreasing the volume. However, weigh your Response Rate as well—if your emails are getting a good response from a committed base of supporters, maybe it's just as well to let those who are less interested unsubscribe, or to try to decrease the volume of emails only for those who aren't as active.
• Allow unsubscribes by type. Provide recipients with an opportunity to unsubscribe from a particular type of email communication instead of broadly opting out of all email communications. A subscriber might appreciate your eNewsletter, but be put off by fundraising appeals.

Mastering Metrics

The four basic metrics you need in order to track and measure the success of online communications (Open Rate, CTR, Response Rate and Unsubscribe Rate) are simple once you’re in the habit. Be diligent—after tracking the results for each email, you will develop an understanding of what response to email campaigns you can expect, and learn to quickly identify if something is going wrong.

Don’t obsess about how your metrics compare with those of other organizations, but focus on improvements or downturns for your own list. As you test new ideas and work to optimize your online communications, try things multiple times to determine what really works with your list.

Are your emails working? Are they worth the effort? With the right email metrics, you can find out for sure. And with all the information you gather, you can fix a failing campaign, or boost a good campaign into a great one.

For More Information

The eNonprofits Benchmark Study [5]
Fabulous, research based look at benchmark figures for metrics like Open Rates, Click- Through Rates, Response Rates and more, based on actual campaigns by large nonprofits.

An Intimate Look at a Successful Online Fundraising Campaign [6]
In-depth case study of how factors like subject line, images, and body text come together in a successful email campaign.

Why Open Rates Are Dropping [7] (PDF)
Useful summary of the causes for the sector wide decline of open rates

Thanks to Alison Cherry of Beaconfire [8], Eve Fox of M+R Strategies [9], and Clarissa Goodlett of Color of Change [10] for their contributions to this article.