The Case for A/B Testing
We all know that fundraising is a nuanced blend of art and science, but what we might not realize is how much of our art as fundraisers is informed by our own personal experiences and perspectives. Many of the decisions that we make are based on gut and instinct (the art), just as much as others are based on research and evidence (the science). Unfortunately, what is moving or eye-catching to us might not necessarily be so compelling to the majority of the people in our intended audience. The kinds of creative decisions we’re talking about include things like which photo to put on the year-end appeal letter, which subject line to use in a fundraising email, or even the graphical layout of an event invitation.
Luckily, with some foresight and planning, you can check your gut decisions and run an A/B test to solve for the areas where you’re not actually sure how to get the best result. Here are five reasons why we think this approach is a must!
A/B Testing has never been easier. Email platforms like Mailchimp and Constant Contact have an A/B Testing feature built right into their systems so that you can test email subject lines, sender names, salutations, and the like. The platforms will randomly sample a small percentage of your email audience, split that sample into two groups and send each of them one version of the element that you are testing for. At the end of 24 hours, the platform will automatically take the winner of the test and send that version of your email to the rest of your list. If you want to conduct a test on something a little more analog, like a letter, there are plenty of free list randomizers on the web that can help you pull a random sample of your mailing list to test.
Even best practices can be made better. Best practices are a great way to guide us in working with broad or general audiences, but your audience is really specific to your organization. You can take general best practices and refine them for your community by testing, for instance, the timing of your appeals, the specific kinds of stories you are telling in them, and the response vehicles you provide.
You can let the data trump instinct. Sometimes when it comes to the creative aspects of fundraising you may have some self-doubt or internal disagreements on your team as to which approaches are best. An A/B test can be a good way to navigate an impasse and make sure that your decisions are being made based on data, rather than on gut.
It forces longer-term planning. Because running an A/B test takes some time, it forces you to think ahead when you’re plotting out your communications with donors. You can take advantage of the opportunity to make sure you’re being thorough in your advance planning where possible so that your plate is clear when the inevitable last-minute fires inherent in nonprofit work start to flare.
The learning about your audience will be invaluable. What you may learn through a series of A/B tests on your donor communications may tell you much more about your audience than how to get the largest percentage of them possible to open an email. With the right tracking tools, you may learn about which kinds of stories and which methods of delivery resonate the most for which of your donors—information that is essential to developing successful donor cultivation strategies.
For A/B testing to be successful, it has to be done thoughtfully and it needs to become a regular part of your process. Done correctly, it can lead you to a refined, effective strategy for communicating with your donors.
For more on donor communications, check out our posts Creating Urgency without Desperation, Getting the Most Out of Your Donor Surveys, and Staying in Touch with Donors: 6 Innovative Strategies.