AB Testing
AB testing or split test is one of the most effective ways to improve conversion rates. Taking an A/B test is a very useful element in many marketing applications and setting up a test is easy.
What is A/B testing?
AB testing consists of distributing two versions of the same content (landing page, email or call-to-action, for example) to two audience samples of similar size, in order to compare performance of each variant.
What element to do an AB test on?
When the company wishes to optimize its web page or a particular campaign, it must test several variables present on the site. The goal to do AB testing is to isolate each variable in order to have concrete results and measure efficiency as well as possible. Here are some interesting variables to test on a content.
Page titles
Changing a few words or the full title of different web pages can influence the interest of Internet users. In fact, the more impactful the message, the more the click-through rate will increase. Testing different headlines on a page allows you to see which one engages the most visitors. In order for the headline to gain more attention from prospects, the company can also change the color and font.
The structure of the pages
The pages of a website must be the subject of particular care in terms of their structure and the hierarchy of content. There are several possibilities for this: modify the banner, the locations of any advertising banners, insert a carousel of images or offer the most interesting products in specific areas.
The text
A / B testing provides invaluable help in defining the main text to be used on the various web contents of a site. In order to get prospects to take action, it is important not only to present a catchy text, but also to think about its place.
Images
Just like text, images occupy an important place in web content. They are therefore part of the essential elements to be tested. It is then a question of working on their dimensions, their luminosity or their positioning within the page.
The call-to-action (CTA)
The call-to-action button, or CTA, is an essential ingredient to influence the conversion rate of a web page. It encourages visitors to take an action: purchase, download, access a form, subscribe to a newsletter. To produce maximum impact, it is important to play with the color, the location and the words used in the CTA.
The forms
A website must have a form that is both clear and short enough to be sure not to lose the user. In order to increase the conversion rate, the company can test the number, location and titles of the form fields.
Algorithms
The use of algorithms can prove to be beneficial in transforming Internet users into buyers, prospects into customers. For example, the company can suggest products to its customer, the most sought after, the most sold or products similar to its initial research. The AB testing phase will thus make it possible to increase the basket and to retain customers as much as possible.
The prices
The idea is not to present the different prices, but rather to reorganize the way they are displayed: location, font, color. The company can also test different offers online to test the impact on the average shopping basket of visitors. Read also: Pricing Strategy | Different pricing types, examples and the Importance to the Success of Your Business
Emails
The A / B testing phases are also consistent on media other than websites. A company can thus send different models of emails to test the responsiveness of recipients. The variations will then relate to elements such as:
the object to optimize the open rate;
content to increase the click-through rate;
the name of the sender.
As with the A / B testing on web pages, the important thing will be to be able to extract statistical data to adjust the next shipments according to the results.
Segmentation and targeting
A/B tests most commonly apply the same variant (e.g., user interface element) with equal probability to all users. However, in some circumstances, responses to variants may be heterogeneous. That is, while a variant A might have a higher response rate overall, variant B may have an even higher response rate within a specific segment of the customer base.
For instance, in the above example, the breakdown of the response rates by gender could have been:
| Gender | Overall | Men | Women |
|---|---|---|---|
| Total sends | 2,000 | 1,000 | 1,000 |
| Total responses | 80 | 35 | 45 |
| Variant A | 50/1,000 (5%) | 10/500 (2%) | 40/500 (8%) |
| Variant B | 30/1,000 (3%) | 25/500 (5%) | 5/500 (1%) |
In this case, we can see that while variant A had a higher response rate overall, variant B actually had a higher response rate with men.
How to do AB testing?
Once the various site or web page variables have been taken into account, it is important to understand how an A / B test works. Good use of A / B testing allows you to make the right decisions to optimize content performance.
The 8 steps to create an A / B test
This article explains how to design, run, and interpret an A / B test to improve conversion rates. This method is applicable to all types of marketing activities. However, for illustrative purposes, it is used here to test a call-to-action (CTA).
Follow the steps below to perform an A / B test.
1 – Select an element to test
A / B testing involves two versions of the same web content, which may differ in subtle or obvious ways. It is possible to test a single variable, such as the color of a CTA, or complex content, such as an entire page.
If the versions tested have many differences, it is the content as a whole, and not its particular elements, which should be considered as the variable to be interpreted. So, if the test looks at two versions of a landing page each displaying different CTAs, forms, images, and headlines, their comparative performance cannot be explained based on just one of these variables. These four elements combined should be seen as a whole.
To improve visitor-to-lead conversion rate, it is recommended that you test landing pages, emails, and call-to-action. The A / B testing example developed in this article concerns the color of a CTA button.
Read also: Business Idea for SME (Small Medium Enterprise)
2 – Define the objective of the test and identify the indicators to be measured
Setting up an A / B test requires thinking about the objectives in advance. For example, it is possible to observe how the color of a CTA influences the click-through rate. This is one of the easiest tests to perform. It can also be to check if the color leads visitors to click on the CTA more than once.
In the example below, the goal is to redirect as many visitors as possible to the landing page. Thus, the indicator measured will be the number of clicks.
3 – Define the reference version and the test version
The benchmark version is Item A of the test – either the landing page, email, call-to-action, or original headline. The test version is item B, which includes modifications.
It represents the status quo. The test version (B) must therefore be different, for example blue in color.
4 – Create the A / B test and distribute it
Once the objective and the indicators have been defined, it is a question of creating the content to be tested. In this example, the only variable tested is the color, the text and the design remaining the same. Indeed, this test aims to measure the effect of color on the number of clicks.
The A / B test must then be configured in the marketing software used. The steps may vary from one tool and one type of content to another.
5 – Promote the tested content to a relevant sample
For the results of a test to be statistically significant, the content tested must reach a relatively large audience. So, a test sample should be carefully determined: an email should be sent to a sufficiently long list, a landing page can be promoted on social media, and a blog post can benefit from a paid campaign.
If the A / B testing targets a specific audience, promotions should be targeted accordingly. For example, if the test is to gauge the popularity of a landing page with Twitter followers, it should only be delivered to Twitter, not Facebook or email.
In the previous example, which looks at conversions associated with CTA, the blog post should be promoted to all audiences that might be interested.
6 – Collect enough data
Now it is a question of showing patience. The promotion of a test must continue until a statistical threshold is reached allowing the assertion that the results are significant, and not the result of chance. Some mathematical formulas make it possible to calculate this threshold. Specific tools are also available. Once the statistical threshold is reached, it is possible to determine whether the test version is more efficient than the reference version.
To obtain statistically significant results, it may sometimes take a month to collect sufficient data. If a test does not produce significant results after 30 days, despite heavy traffic, it may mean that the tested variable has limited impact. In this case, the test can be interrupted.
7 – Extend the analysis to the entire marketing funnel
Once the targeted indicators have been measured, the analysis should be broadened. This is to check if the test has produced effects on other segments of the funnel.
Something seemingly innocuous, like the color of a CTA, can have an impact beyond click-through rate. Closed-loop analytics, for example, make it possible to check whether visitors who clicked on the CTA have converted into customers. Maybe the blue CTA spawns customers faster than the gray version.
This is probably not the case in this example, but more complex A / B tests can have such effects. So looking at the entire funnel sometimes reveals unexpected results. Some may be of commercial interest, while others may question the appropriateness of the proposed changes. It is therefore important to keep in mind that the impact of A / B testing may exceed its initial objective.
8 – Apply the conclusions
Data has been collected and results analyzed across the funnel, but the job is not finished. A first A / B test often opens up new questions. In the example, in addition to the color of the CTA, it is possible to test its placement or its text.
However, sometimes the results of an AB testing are not convincing. This may have been carried out during a peak in seasonal traffic and is therefore not representative of usual visitor behavior. In this case, it is possible to repeat the test at another time.
Analyze an AB test
results of an A / B test carried out with HubSpot
Analysis is a delicate, but crucial step in A / B testing. The use of this technique must include a reporting interface displaying the following indicators:
- the number of conversions recorded per variable,
- the conversion rate,
- the percentage of improvement,
- the statistical reliability index for each variable.
In order to analyze the behavior of prospects and customers, it is also possible to segment the results of the A / B test into different categories:
- Age,
- Geographical area,
- Source of traffic,
- New visitors,
- New buyers,
- Subscribers, etc.
This type of information makes it possible to reorient or act on future marketing actions.
To analyze at best, the company must also think about measuring the reliability of its test according to the traffic gained by its website. A / B testing is considered reliable if the confidence level is 95% and the statistical power is at least 80%. It is also important to analyze which pages generate the most traffic.
AB testing tools
Many tools allow you to perform tests on a page or a website. Here are some interesting solutions for an A / B testing campaign:
Comparison of A / B testing tools
Below are commons AB testing tools:
| Tool name | Company | Platforms | Tests | Email campaigns | Interface | URL |
|---|---|---|---|---|---|---|
| AB Tasty | AB Tasty | Web et mobile | ✓ | ✗ | Graphique | [1] |
| BeamPulse | BeamPulse | Web | ✗ | ✗ | Graphique | [2] |
| Convertize | Convertize | Web | ✓ | ✗ | Graphique | [3] |
| Devatics | Devatics | Web et mobile | ✓ | ✓ | Service | |
| Google Optimize | Web | ✓ | ✗ | Graphique et API | ||
| Kameleoon | Web | ✓ | ✓ | Graphique et API | [4] | |
| Maxymiser | Oracle | Web et mobile | ✓ | ✓ | Graphique | [5] |
| Optimizely | Optimizely | Web et mobile | ✓ | ✗ | Graphique et API | [6] |
| Visual Website Optimizer | Web | ✓ | ✓ | Graphique | [7] | |
| Webtrends Optimize | Web | ✓ | ✓ | Graphique et API | [8] |
HubSpot, which allows testing of website pages, CTAs, forms or emails with the Pro version of the Marketing Hub.
Google Optimize, via Google Analytics, which allows up to 10 free test versions of a single content to compare their results by user segmentation.
Convertize, A / B Testing tool available in French and accessible for people without the technical skills of a webmaster. From € 39 per month, with a 14-day free trial.
AB Tasty, a paid optimization solution that is ideal for medium-sized businesses. The tool makes it possible to test the level of visitor engagement and increase traffic to the website concerned. Rates not communicated.
Optimizely, the most famous American solution allowing to perform several experiments on the same page. The goal of this tool is to provide the best user experience. Paid tool, prices not communicated.
VWO or Visual Website Optimizer, a tool offering different functionalities to evaluate the efficiency of A / B tests. Starting at $ 199 per month, with free trial.
Kameleoon, a paid French tool for testing or customizing all of the content of a web page in order to boost conversions. Rates not communicated.
AB test calculator. The significance calculator will tell you if a variation increased your sales, and by how much. You can try: AB Test Guide atau Neil Patel
Performing regular tests is an effective way to optimize conversion rates in a sustainable and controlled manner.
To go further, download these marketing experience templates and set up a process that will support the growth of your business.
Sources: PinterPandai, Techopedia, Kameleoon, Mailchimp
Photo credit: Pixabay