The Ultimate Amazon Feedback Email Experiment: How A/B Email Testing Can Improve Amazon Product Reviews and Seller Feedback
- Lena R. Liberman
- On May 27, 2020
Table of Contents
- What Is A/B Testing?
- Why Run an Amazon Feedback Email Test?
- Finding Winners: How to Create and Run an Amazon Feedback Email Test . . . and Leverage the Discoveries
- Collect the Data and Then Let It Guide You
- Why Use Emojis in Your Amazon Buyer-Seller Emails?
- Tips for Using Amazon Emojis in Email Subject Lines and the Body of Emails
- How to Add Emojis in Seller Labs Pro Emails
Chapter 1: Using A/B Email Testing for Better Amazon Buyer-Seller Messaging
What Is A/B Email Testing?
A/B testing, also known as “split testing,” allows you to create and run two comparison messages and analyze email opens, click-throughs, and conversion rates to determine the better performer. Within the Seller Labs Pro Communication Center Powered by Feedback Genius, A/B testing works alongside message analytics to help you identify and optimize the message content and configuration that best helps you achieve your communication goals with regard to seller feedback, product reviews, and customer service.
This is a great feature if you’ve been using Seller Labs Pro or Feedback Genius for a while and you want to further optimize your account, or if you’re using Buyer-Seller Messaging within Amazon’s Seller Central and you want to create more-sophisticated, more-effective messages than what’s available there. With A/B email testing you can improve both your messages and your business by using data to boost conversion rates and better engage with buyers.
Why Run an Amazon Feedback Email Test?
A/B testing tells you how well your messages are performing and it provides opportunities for improvement.
Say that you have a message sequence that requests seller feedback or product reviews from your customers and your conversion rate for that sequence has dropped. Obviously you’d wonder why that’s happened and how you can boost conversions.
You could change the subject line or edit the message itself and change how you make the request, but in doing so, you’d lose your original message and be left with no benchmarks,
Instead, keep the original message, duplicate it, and using that duplicate, create a slightly different counterpart message using A/B testing so that you can compare which of the two messages yields the highest conversion rates. With this A/B testing method, there’s no guessing which message is more effective. You have your answer because you’ve run the two messages head to head in your own Amazon feedback email test scenario and the results will provide the data that tells you which message works best.
A/B email testing for buyer-seller messaging is not available in Amazon Seller Central. To reap the rewards of split testing, you’ll need to use the Seller Labs Communication Center Powered by Feedback Genius, just one of the many tools within Seller Labs Pro (start your free 30-day trial of Seller Labs Pro today). A/B testing is just one of the many powerful Pro features that goes beyond the basics and limitations of Seller Central.
Finding Winners: How to Create and Run an Amazon Feedback Email Test . . . and Leverage the Discoveries
Within the Communication Center, start by creating an A message and a B message, essentially two counterpart messages that you want to compare. These messages can have different message content, different subject lines, and they can even be sent at different times. You might try creating two copies of one message and simply changing one thing between them so as to isolate a single variable and see how the different messages perform, or you could run two entirely different messages and see if one approach really moves the needle more than the other.
Whatever you do, remember to keep it apples to apples and compare only messages that are trying to achieve the same result. There is no sense comparing one message that is asking for a product review and using emojis in the subject line with one asking for seller feedback with no emojis.
Next, divide your orders based on the numbers contained in your customers’ order IDs. For example, in message A, you would enter in 12345 and that message would only be sent to orders ending those digits. By the same token, in message B you would enter in the remaining numbers (67890 which would be the other orders). You can also divide up your orders by even or odd numbers. For example, message A would be sent to orders ending in even numbers 24680 and message B would be sent to orders ending in 13579. You do you, just be consistent and balanced in your splits.
Collect the Data and Then Let It Guide You
After splitting your messages by order number and doing a few weeks of sending, you can check your message Analytics to see which message yielded high conversion rates. You can check as frequently as every week, but we recommend letting the A/B Amazon feedback email test continue for at least a month to get a clear idea of your message’s performance.
A/B testing is a powerful tool that can help you optimize your Amazon seller account to yield higher conversion rates for product reviews, seller feedback, or even just to know that your messages are getting noticed by your customers. You can even become more proficient in your messaging tone and better establish your brand’s voice so you can speak to customers in a way that resonates with them and invites them to be brand advocates and repeat buyers.
Chapter 2: Increase Your Communication Impact with Amazon Emojis
Use popular emojis in your subject lines in order to increase open rates for your Amazon emails.
You’ve seen emojis everywhere. You probably use them during texts with your friends and family. But have you ever thought about using Amazon emojis in A/B email testing with your buyer-seller messages? Try adding your choice of more than 1,000 different emojis to your Amazon emails and subject lines to visually tell a story and make your messages stand out.
Why Use Emojis in Your Amazon Buyer-Seller Emails?
Emojis are visual representations (ideograms) that are universally understood regardless of language. Essentially, they let you say more with less and achieve greater understanding globally. Emojis in business emails have risen in popularity over the past couple of years. Recent data shows that 56% of brands using emojis in their email subject lines had a higher unique open rate.
With the normalization of emojis in our everyday communication, it only makes sense to use them in your Amazon emails — or at the very least, testing them in your emails. Emojis can help your emails stand out in crowded inboxes.
Tips for Using Amazon Emojis in Email Subject Lines and the Body of Emails
- Understand Your Audience: When it comes to using emojis in your emails, it’s critical that you understand your audience. Chances are you’ll already have a general idea of what resonates with your buyers. Even if your audience tends to be more serious than others, there’s likely an emoji that will suit your needs. Avoid certain emojis and never risk losing a customer by using a potentially offensive emoji. In a nutshell, unless you’re selling diapers or something of that ilk, maybe skip the poopmoji.
- Connect Your Message to the Emoji: An emoji should match, complement, or enhance the text in your subject line and email body. For example, if you’re sending an email related to the holidays, using the snowman, snowflake, or Christmas tree emojis would match the context of your message. Sending an email about a product tied to the holiday season with the beach umbrella emoji isn’t relevant to your message — unless it’s about getting away for the holidays.
- Convey Excitement and Emotion: Faces and hands are the best way to communicate an emotional action with an emoji. That’s because our faces and hands are the most expressive communication tools we have. There are about 200 emojis available to communicate emotional actions. Generally, positive and colorful emojis will be more effective at capturing attention.
- Be Literal and Let Eye-Catching Symbols Do the Work of Words: If you sell a piece of equipment that has an associated emoji, use that emoji to do the heavy lifting! Let’s say you sell footballs on Amazon. Don’t you think adding the football emoji to your subject line will be eye-catching and representative of your product? For example, you could write: “What do you think about your new football?”or “What do you think about your new 🏈?” It’s pretty clear which subject line stands out more. Plus, you’ve saved characters and space, which means better rendering on mobile devices.
- Create a Narrative: This tactic is going to take a bit more creativity than being literal. But you can use emojis to create a short narrative. For example, if you sell natural cat food with fish-based omega-3 oils, you could use a sequence of emojis like this 🐟 + 🐈 = 😺 to communicate how your product makes cats happy.
How to Add Emojis in Seller Labs Pro Emails
Within the Messages tab in the Communication Center Powered by Feedback Genius, you can edit an old email or create a new one. In the message editor, click the smiley face icon to browse through the eight sections containing the 1,000+ different emojis. Upon clicking an emoji, it will appear in the body of your message. It’s that easy.
If you want to add emojis to your subject line, just write out your subject line with your desired emojis in the email body then cut and paste it in the subject line section above. Now you can test your new emoji-enabled subject line against a previous or text-only subject line to see which one is more effective.
Chapter 3: Using A/B Email Testing to Optimize Buyer-Seller Messaging Open Rates
If one thing about selling on Amazon is certain, it’s that nothing about selling on Amazon is static or fixed. The challenges posed by frequent changes to Amazon policies and within consumer behaviors make it nearly impossible for sellers to predict what’s next or how to prepare for it. But the same challenges also present great opportunities for sellers who are nimble and who understand how making data-driven choices can lead to steady growth and picking up what the competition fails to collect. This is the essence of real optimization. A great place to start is by achieving dramatic, positive changes by using A/B testing to optimize your Amazon buyer-seller messages open rates.
Optimization Always and in All Ways
Just as you should be looking to optimize your product listings using keyword discovery tools and to optimize your Amazon Sponsored Advertising using PPC tools with intelligent suggestions, so should you be looking to optimize your customer communications using automated buyer-seller messaging. And I know that many sellers are doing just that and to excellent effect, but are you optimizing how you optimize?
You’re probably familiar with Chad Rubin of Skubana. Chad’s a long-time Seller Labs partner and a Seller Labs Pro user who’s had great success getting reviews and feedback with the program. Our Communications Success Team reviewed Chad’s account to identify areas where he could elicit even better results. Because there are always tests to run and settings to tweak and data to gather, we indeed found some areas where we thought Chad might optimize. Here I’ll focus on A/B testing with regard to email open rates, just one of the areas where sellers, including power sellers like Chad, can get even better.
Case Study: Improving Buyer-Seller Messaging Open Rates
Scenario: We looked at open rates for two of Chad’s messages (“General Feedback” and “Warranty”).
Recommendation: In early November, as a result of recommendations by the Seller Labs Communications team, Chad rolled out A/B testing wherein he tried two variations of each message’s subject line in order to see if one performed better than the other in terms of opens (remember that you can also do A/B testing with entire messages, not just subject lines).
Result: During this time, a couple of things happened:
- Chad’s overall open rates increased for both the General Feedback and the Warranty emails as both versions of both subject lines performed quite well. Well done, sir!
- Chad got data indicating which of his subject lines (original message A or new message B for each message) got the better open rates. In these cases, the numbers indicated that the original subject lines Chad had been using performed a bit better than the new ones he was testing. That’s a great find as it confirms that Chad’s been using subject lines that work. But are these the best subject lines that he could be using? Not necessarily, which is why our team recommended revisiting A/B testing again in the future, this time with still-different subject lines that Chad could run against his originals.
Running an Amazon feedback email test is easy, and fast, and powerful. It can yield amazing insights that enhance your customer communications and get you more and better reviews and feedback.
Chapter 4: Lessons, Key Takeaways, and Next Steps
- Revisiting, even revisiting what seems to be working, in pursuit of even-better results is the essence of optimization. Be it with your product listings or your Sponsored Products ads or your buyer-seller message subject lines or body content, you can always test and find ways to improve. Remember that what works best today is not what will work best tomorrow, so test again and in new ways always.
- Don’t be discouraged if your improvement is only by a small percentage. That’s still an improvement and there is gold in the margins. Over time, enough small improvements add up to a big difference.
- Never ever stop testing, trying, and pursuing even incremental improvements. There is always a better way to be found through testing and fortune favors those who aren’t afraid to listen to the data speak and to act upon it. Or, as our friend, seven-figure seller and Seller Labs Pro user Liran Hirschkorn, put it so well:
“I constantly want to be testing everything to see what could be working better. In internet marketing, this is called split or A/B testing. With the Seller Labs Pro Communication center Powered by Feedback Genius, I can choose to send half of my customers one email, and the other half a different email, and then test which one worked better. For example, if I want to increase my email open rates, I can start by leaving my entire email sequence the same and only change my subject lines. In one subject line, I can test ‘How do you like your garlic press?’ while simultaneously testing ‘Here’s a simple way to clean your new garlic press’ in another sequence.”
You Owe It to Yourself to Find Your Best Techniques
If you’re ready to see what A/B testing can do for your open rates and overall buyer-seller messaging, all you need is a Seller Labs Pro account. If you don’t have one yet, get a free 30-day trial and start improving all aspects of your Amazon game.
Or, if you’d prefer to leave A/B testing and your Amazon buyer-seller messaging optimization to the experts, check out Seller Labs Managed Services Amazon Buyer Seller Messaging Optimization.