We are going back to the basics this week on the blog. We are going to review an oldie but a goodie, and how you can use it to possibly help raise your conversion rates!
A/B testing, or split testing, is a concept that’s been around since the early 1920s. Going into the 60s and 70s, marketers took the concept and applied it to marketing projects and ran with it. Since then, A/B testing has been one of the most basic concepts business owners have used to develop successful marketing tactics.
What is A/B testing?
A/B testing is a concept where you compare two versions of something, comparing method A to method B, and determining which version did better. This is important when you’re in the first stages of planning out a marketing strategy and not sure what method to use.
We have reviewed what an A/B test is, but how would we conduct one? Let’s review that next.
How Do We Conduct An A/B Test?
The first and probably most crucial step is to determine what you want to test. Are you wanting to send out a more frequent newsletter or try out a new format for your website? It could be something as simple as the size of the CTA (call to action) icon you want them to click.
The next step is determining how you will gauge the performance of each method. For the newsletter scenario, maybe you want to see if you get more website traffic from sending a more frequent newsletter. Looking at the formatting of a new website, does the consumer stay longer on the website? For the size of the icon, does a bigger icon mean more clicks? We want to focus on how the data could help us in determining our next step.
To run the test, we need participants. The key in choosing participants is to keep it random. A/B testing is built on the idea of a randomized but controlled experiment. Just like in a medical study, participants are randomized, one group gets the placebo and the other gets the actual drug being tested. In this way, the participants won’t know if they are taking the drug or not. Then their responses cannot be compromised with prior knowledge.
Now we know what we are testing, why we are testing it, and that it will be a random sample of participants. The next step is to determine what sample size will be most statistically significant for the test. We want a big enough sample size to make a difference in the data we collect.
When determining how long to run your A/B test, there is a balancing act between the test being too short or running into convergence. This is a fancy way of saying that in the beginning the results of the two vary, strongly favoring one method but then that difference decreases over time. If you happen to run into convergence, you can deduce that the two methods don’t have a lot of significance in what you’re trying to learn from the test. It is best practice to have the test run for several weeks to make sure you get good results.
How Do We Analyze and Use the Data?
The test has concluded and now you have all the data that you need to organize. The easiest way to display the interpreted data is a percentage. Which method performed the best and by how much?
Organizing the data and putting it on paper, helps the reader compare the two better and analyze the outcomes. Were the percentages far apart? Did method A barely outperform method B? One could infer that the difference between the two was not significant enough to make much of a difference.
Once we have the data organized in front of us, we can determine how to utilize this information for future marketing efforts. Did the larger button have more clicks? We now know that a larger call to action icon leads to more clicks, so we know to utilize this in emails in the future. A/B testing can help marketing efforts in many ways and is a great tool to use to determine what marketing efforts are effective before releasing them to the public.
Conclusion
As a wrap-up, let’s go over what we learned, or relearned. We understand that A/B or split testing is a method of comparing two different ideas together to see if one is better than the other. We can test multiple marketing tactics like the format of a website, the size or color of a call-to-action button, etc. We went over the steps on how to conduct an A/B test which are:
Decide what to test on
Determine your end goals
Choose participants, keeping it random!
Determine the sample size you will need
Determine how long the test should run
Run Test
Collect data
Organize and analyze data
We also discussed different ways we can interpret the data and how that data is important to future marketing efforts. All in one blog, phew!
As always, if you have questions or would like some marketing advice, you can always contact us at Elemental Plans. We are your marketing experts and have helped many in the past determine marketing objectives and goals and how to attain them. You can fill out this short form and we will get in contact with you as soon as we can!
Sources
Harvard Business Review, “A Refresher on A/B Testing”, Amy Gallo, June 28, 2017, https://hbr.org/2017/06/a-refresher-on-ab-testing, Accessed November 19, 2022
The Daily Egg, “How to do A/B Testing and Improve Your Conversions Quickly”, Today’s Eggspert, June 29, 2021, https://www.crazyegg.com/blog/ab-testing/, Accessed November 19, 2022
留言