Creating Effective Email Split Tests

posted on 7/6/2016

Email is one of the most effective marketing channels, but because there’s almost infinite room for variety it can be difficult to determine what works for your business. Split testing can help you fine-tune your email strategy to maximize its effectiveness.

Split testing, also called A/B testing, allows you to test a single element of an email by sending different versions to different segments of your subscriber list to see which version performs best. The results can give you a wealth of information about how your customers want to receive information from you.

To make sure you get the most accurate results, use these best practices:

• If you use an email marketing platform such as MailChimp or Constant Contact, use their built-in split-testing tools. If not, divide your email list (or a percentage of it) randomly into two equal A and B groups.
• Create a clear hypothesis to test. Be as specific as possible about what you’re testing, what result you expect, and why.
• Only test one thing at a time. If you’re testing a call to action (CTA) button, for example, don’t change the color, text, and location of the button for a single test. Test each aspect individually then use the winner in subsequent tests.
• Send your test email to each group simultaneously unless you’re testing for send time.
• Use existing data to inform your tests. If you always see strong open rates when you mention a discount, create a specific test to prove or disprove your theory.
• Bear in mind that the end goal of all tests should be to create meaningful results to improve your business.
• To determine the winner, check key indicators including open rate, click through, and conversion rate.

Here are some A/B test ideas to get you started:

• Call to Action
  Making sure you have the right CTA can produce quick, big wins for your business. There are a lot of variables just within this category—you can test conversion language (Buy Now, Purchase, Checkout, Add to Cart), color, shape, placement (top of page vs. bottom of page), one CTA vs. multiple, text vs. icons, hyperlinks vs. buttons, etc.
• Send Time
  Do your customers read your emails first thing in the morning or wait until after hours to review them? Are they more receptive on Mondays or Thursdays? Testing for time of day and day of week can tell you when they like to hear from you.
• Subject Line
  Subject lines are a major factor in how many of your emails get opened, so accurate A/B tests can yield significant results. Test for length, tone, capitalization, emojis or symbols, keywords, questions vs. statements, and more. (For more on subject lines, see Subject Line Hacks to Get Your Emails Opened.)
• Design
  If you think your email might benefit from a visual refresh, create a new design or format using the same exact content to see which receives a more enthusiastic response from recipients. Alternatively, you may want to try a simplified design with fewer images to see if that affects open rates. 
• Image
  What do your customers respond to visually? You can test images of people vs. landscapes, photos vs. illustrations, images vs. no images, static image vs. video, product photo vs. 360° product image.
• Sender Name
  Do your subscribers respond better to a generic email return address (newsletter@companyname.com) or do they feel more comfortable seeing a name they recognize (mary@companyname.com) in the From field?
• Content
  Try content in paragraph form vs. a bulleted or numbered list, short vs. long paragraphs, negative vs. positive messaging, serious vs. lighthearted tone.
• Personalization
  Try using a personalized greeting in the body of the email. An email that opens with "Hi, Mary!" might get a better reaction than one with a greeting such as "Dear Customer." 

Split testing is an ongoing process, and though it takes time and attention, it can be invaluable way to make sure your emails are working to your best advantage.