Does Direct Mail Work?
Intro
In 2012, the Colorado Republican Party conducted a groundbreaking study on the effectiveness of political direct mail that was ultimately published in Political Research Quarterly. Since it’s that time of year where everyone is preparing the paid media for their campaigns, let’s answer the question — does direct mail actually work?
Direct mail, positive or negative, can increase name ID for the candidate mentioned.
Positive mailers do increase candidate favorability. While negative mailers do appear to harm a candidate’s favorability, it’s not a significant impact.
Direct mail does increase voters’ stated intention to vote, but not their actual voting behavior.
The impact of direct mail fades rapidly.
Direct mail reaches saturation quickly and loses all effectiveness by mid-October.
Background
I bet you didn’t know it, but the Colorado Republican Party is one of the leading authorities on direct mail.
In 2012, data-driven campaigning was really coming into its own. We were breaking through the barrier into the data-science age, and quantitative research in campaigns was all the rage. Many of you may have read the two books that captured this interest — Rick Perry and His Eggheads and Get Out the Vote!
At the time, Chuck Poplstein and I were both still with the Colorado Republican Party and we took a keen interest in this new, more precise approach to campaigning. It was the first year under the current legislative map, there were a number of competitive legislative races at play, and it was also the first year of the State Party’s Trailblazer program to support legislative candidates.
Chuck and I got to wondering:
“Was all of this direct mail that the State Party was spending money on actually doing anything?”
Most research on the impact of different campaign activities had only addressed turnout, not persuasion. Chuck and I set out to change this.
We had the resources and the know-how to design and execute a sizable direct mail program, and even to test it. But, to ensure that our results withstood scrutiny and to add credibility to the project, we partnered with two professors — David Doherty and E. Scott Adler. They helped ensure that the experiment would produce accurate results and that it would stand up to scrutiny.
Doherty and Adler went on to publish the study in Political Research Quarterly. At the time it was only the second study of its kind ever done and it became the official word on the impact of partisan direct mail.
Setup
To answer our various questions about the impact of direct mail we selected three competitive Trailblazer campaigns:
SD19 – Lang Sias
SD26 – Dave Kerber
SD35 – Larry Crowder
We conducted two rounds of trials — one around August 21st and the other around October 16th. We randomly assigned members of the target universe of Unaffiliated voters for each race to four different groups, those who would:
Receive a positive mailer about the Republican.
Receive a negative mailer about the Democrat.
Receive a contrast mailer with a positive message about the Republican on one side and a negative message about the Democrat on the other.
Receive no mail to act as our control group.
Using this design we set out to determine:
The impact on candidate name ID.
The impact on candidate favorability.
The impact on turnout.
How long the impact of a mailer persisted.
When saturation set in.
The direct mail pieces were pretty standard political fodder. Here is an example of the pieces sent out in support of Kerber and Sias:
Results
In short, we found:
Direct mail, positive or negative, can increase name ID for the candidate mentioned.
Positive mailers do increase candidate favorability. While negative mailers do appear to harm a candidate’s favorability, it’s not a significant impact.
Direct mail does increase voters’ stated intention to vote, but not their actual voting behavior.
The impact of direct mail fades rapidly.
Direct mail reaches saturation quickly and loses all effectiveness by mid-October.
Name ID
The most encouraging result was that direct mail does, indeed, increase candidate Name ID by a significant amount. This is true whether the candidate is mentioned in either a positive or negative light. In fact, the increase in name ID is actually greater with a negative message. This suggests that voters retain negative information at a higher rate.
In our first, August 21st, test we found that a positive mailer increased name ID for the Republican candidate by 4.7 points. The negative mailer increased name ID for the Democrat candidate by 5.2 points.
In our second, October 16th, test there was no discernable impact on candidate name ID in neither the positive, negative, or contrast instances — an outcome I’ll discuss further in the timing section below.
Persuasion
The results from our first trial in August showed that mailers had the intended effect on candidate favorability, albeit to a much smaller degree than is conventionally assumed.
The positive mailer increased the favorability of the Republican candidate by about one-fifth of a standard deviation. This effect is small but still rises to the level of statistical significance.
The negative mailer did seem to hurt the favorability of the Democrat but the results were not statistically significant.
We also measured the relative standing between the two candidates’ favorability. In this case, both positive and negative mailers appeared to improve the standing of the Republican but neither result was statistically significant.
During the second test around October 16th we found no significant impact on candidate favorability. Also, by mid-October, the impact from our first round of mail during August had vanished entirely.
Turnout
The turnout findings were the most profound and even led to the subsequent development of a new method of polling — which is why Constellation polls are so accurate …but, I digress.
The mailers in both August and our second test in October increased voters’ stated intention to vote. In other words, voters who received any of our test mail said they were more likely to vote.
However!
…when comparing the results against actual ballots cast, we found no actual impact on turnout.
In short, receiving political mail made voters more likely to say they’d vote but they didn’t actually do it.
Timing
If you’re a candidate or any entity looking to do political mail this is the most important finding:
At the level of a competitive legislative race in a Presidential election year, direct mail loses all effectiveness by mid-October.
The intuition behind this result makes sense; as Election Day approaches, the total amount of campaign spending increases and voters begin to, one, ignore political advertising and, two, solidify their opinions about the candidates.
These effects mean that low-impact mediums of advertising lose effectiveness. At later stages of the election, we need higher impact mediums of advertising, like video, to cut through the noise.
This diminishing impact of direct mail started to get more attention after the 2018 election where our legislative losses underscored Republicans’ overreliance on direct mail. I discussed this disparity in my summary of 2018 election spending. Additionally, many of you likely recall this graph on diminishing returns to advertising from the, more closely held, post-mortem of that election.
Is Direct Mail for You?
The takeaway from this study is not “don’t do direct mail.” Rather, it should be: “consider these factors when determining whether or not direct mail is right for you.”
What’s your landscape?
This test was done on competitive legislative races so in one instance there’s an easier answer:
If your race is of competitive legislative or greater salience, your direct mail would need to be arriving in mailboxes right now to have any impact. Moving forward it will only become less effective and by October it will be meaningless.
For those of you in different types of races, take these results into consideration alongside the complete picture of your race.
Consider the amount of total advertising likely to be deployed in your race. Is it likely that other advertising will drown out your direct mail? Will there be high-impact advertising, like TV, for your race that could also drown out mail?
If you’re in a small or low-salience campaign, the great news from this study is that direct mail does have an impact and that a single piece can boost your name ID by five points …if it’s not crowded out by other advertising.
What’s your budget?
You also need to consider your budget relative to the size of your target universe of voters.
When comparing the expense of different forms of advertising a common metric is Cost-Per-iMpression (CPM). This is the amount you pay for every 1,000 “views” of your ad. The below table shows the rough CPM for different forms of advertising:
A lot of factors go into determining the CPM for a particular medium. Table 1 shows that high-impact or targeted mediums tend to be more expensive.
For example, the CPM for autodials is only around $30 — but everyone hates autodials. Contrast to cable at $75 – $250 CPM where you can deploy compelling creative to a captive audience.
Google Ads are very cheap on a per-impression basis because you can blanket the internet with them, but they’re now untargeted. Compare the price of Google Ads at $6 CPM to Facebook ads at $12 – $25 CPM. Facebook is more expensive because those ads can be targeted to specific voters. In both cases, note how much cheaper digital is than other mediums of advertising. That’s because digital is a low-impact medium, it takes a lot more digital to move the needle than it does TV.
Let’s look at these costs in greater context. Table 2 shows how expensive it would be for a competitive Colorado State Senate campaign and a competitive State House campaign to reach every member of their target persuasion universe with one view of an advertisement via each medium.
As you can see, direct mail is far-and-away the most expensive medium of advertising. And we all know that you can’t just reach voters with one ad, one time, and expect to win. This means that for mail alone a competitive State Senate race would need about $100,000 and a competitive State House race would need about $40,000.
In considering which ad mediums are right for you, first consider those you can afford to mount an effective campaign with.
Conclusion
The combined cost and oversaturation of other advertising mediums means direct mail is best suited for campaigns with smaller numbers of voters and/or those that will have minimal advertising from all sources — candidate, opponent, and outside groups.
