The Complete Guide to A/B Testing

What is A/B Testing?

A/B testing (sometimes called split testing) is comparing two versions of a web page to see which one performs better. You compare two web pages by showing the two variants (let's call them A and B) to similar visitors at the same time. The one that gives a better conversion rate, wins!
All websites on the web have a goal - a reason for them to exist.

• eCommerce websites want visitors buying products
• SaaS web apps want visitors signing up for a trial and converting to paid visitors
• News and media websites want readers to click on ads or sign up for paid subscriptions

Every business website wants visitors converting from just visitors to something else. The rate at which a website is able to do this is its "conversion rate". Measuring the performance of a variation (A or B) means measuring the rate at which it converts visitors to goal achievers.

Why Should You A/B Test?

A/B testing allows you to make more out of your existing traffic. While the cost of acquiring paid traffic can be huge, the cost of increasing your conversions is minimal. To compare, a Small Business Plan of Visual Website Optimizer starts at $49. That's the cost of 5 to 10 Google Adwords clicks. The Return On Investment of A/B testing can be massive, as even small changes on a landing page or website can result in significant increases in leads generated, sales and revenue.
If you want to see proof of how others in your industry have benefitted from A/B testing, have a look at our Case Studies page. We've collected more than 70 examples of how online businesses have optimized key metrics through A/B testing.

What Can You Test?

Almost anything on your website that affects visitor behavior can be A/B tested.

Some elements that you can easily
test are:

1. Headlines
2. Sub headlines
3. Paragraph Text
4. Testimonials
5. Call to Action text
6. Call to Action Button
7. Links
8. Images
9. Content near the fold
10. Social proof
11. Media mentions
12. Awards and badges

Advanced tests can include pricing structures, sales promotions, free trial lengths, navigation and UX experiences, free or paid delivery, and more.

A/B Testing and SEO

Google cleared the air on the SEO implications of A/B testing in their blog post titled "Website Testing And Google Search". The important bits from that post are:

No Cloaking

Cloaking - showing one set of content to humans, and a different set to Googlebot - is against our Webmaster Guidelines, whether you're running a test or not. Make sure that you're not deciding whether to serve the test, or which content variant to serve, based on user-agent. An example of this would be always serving the original content when you see the user-agent "Googlebot." Remember that infringing our Guidelines can get your site demoted or removed from Google search results - probably not the desired outcome of your test.

Use 302s, not 301s.

If you're running an A/B test that redirects users from the original URL to a variation URL, use a 302 (temporary) redirect, not a 301 (permanent) redirect. This tells search engines that this redirect is temporary - it will only be in place as long as you're running the experiment - and that they should keep the original URL in their index rather than replacing it with the target of the redirect (the test page). JavaScript-based redirects are also fine.

Only run the experiment as long as necessary

The amount of time required for a reliable test will vary depending on factors like your conversion rates, and how much traffic your website gets; a good testing tool should tell you when you've gathered enough data to draw a reliable conclusion. Once you've concluded the test, you should update your site with the desired content variation(s) and remove all elements of the test as soon as possible, such as alternate URLs or testing scripts and markup. Remove all elements of the test as soon as possible, such as alternate URLs or testing scripts and markup.

The A/B Testing Process

The correct way to run an AB testing experiment (or any other experiment for that matter) is to follow the Scientific Method. The steps of the Scientific Method are:

1. Ask a question: "Why is the bounce rate of my website lower than industry standard?"
2. Do background research: Understand your visitors' behavior using Google Analytics and any other analytics tools running on your website.
3. Construct a hypothesis: "Adding more links in the footer will reduce the bounce rate".
4. Calculate the number of visitors/days you need to run the test for: Always calculate the number of visitors required for a test before starting the test. You can use our A/B Test Duration Calculator.
5. Test your hypothesis: You create a site wide A/B test in which the variation (version B) has a footer with more links. You test it against the original and measure bounce rate.
6. Analyze data and draw conclusions: If the footer with more links reduces bounce rate, then you can conclude that increased number of links in the footer is one of the factors that reduces bounce. If there is no difference in bounce, then go back to step 3 and construct a new hypothesis.
7. Report results to all concerned: Let others in Marketing, IT and UI/UX know of the test results and insights generated.

Your First A/B Test

Starting conversion optimization with Visual Website Optimizer is incredibly easy. Essentially, it is just four simple steps.

1. Include the Visual Website Optimizer code snippet in your website

Including the code snippet means we are now ready to run the tests you create on your website. For further ease, we have plugins for Wordpress, Drupal and Joomla that make the process hassle free.

2. Create variations using the WYSIWYG Visual Editor

Load your website in the Visual Editor and create any changes using the simple point-and-click interface. Advanced users can even make CSS and JS code changes.

3. Choose your goals

All A/B tests have goals whose conversion rate you want to increase. These goals can be straight forward (clicks on links, visits page) or could use advanced custom conversion code.

4. Start and track your test

And that's it, your test is ready to go live. Reporting is real-time so you can start seeing reports as soon as visitors arrive on a live test.

A/B Testing Success Examples

201% increase

Redesigning category webpage increases leads generated

Majestic Wines revamped their category page design to increase online enquiries for their Wedding services by 201%.

144% increase

A/B testing between different pricing structures increases revenue by 114%

Server Density A/B tested between per unit and packaged pricing plans. The winning plan reduced free signups but increased the Average Order Value (AOV), and consequently revenue by 114%.

111% increase

Redesign of ecommerce product page increases conversions

Conversion Optimization Agency Trinity Insight used Visual Website Optimizer to test a better version of the ecommerce product page. This led to a 111% increase in conversions.

15 eCommerce A/B Testing Ideas to Make Your Cash Registers Ring Faster Than Ever

If you love A B testing, eCommerce sites should be your favorite playground. These sites have so much happening at every level that you can never test too much.
Amazon, the market leader, is well known for its habit to test everything. Following its footsteps, when you run short of ideas, here is a checklist for 15 eCommerce A/B testing ideas you can save for reference:

Homepage Test Ideas

1. Add Trust/Award Badges

Many sites have started adding the Trust Pilot customer reviews widget or badge on different steps of the funnel. Express Watches used it on their homepage which increased their sales by 58.29%. You can see how Webtogs display the Trust Pilot Badge on their homepage with other award badges:

Although this looks great, I think a more prominent position of these badges might help improve conversions for Webtogs.
Our customer, Bag Servant, also increased their conversion rate by 72.05% when they added the esteemed “WOW Winner” badge, won by them, in the header of their homepage.

2. Test Your Drop-down Menus

One of our clients replaced its drop-down menu with an elaborate product category page for better usability. Result? Their revenue increased by 56.43%. Read the complete case study here.
Eye tracking studies have also shown that drop-down menus can annoy users. You should especially consider this option if your target audience is from older age group or users with disabilities.

3. Tell Them If You Ship to Their Location

I came across this website named, Threadless.com, while randomly surfing the Internet. I would have never explored it any further if their message “Yep, we ship to India” hadn’t been displayed. I would have thought that it’s probably a US/UK based site that doesn’t have any shipment policy for India (like most websites).

As a Visual Website Customer, you probably know that setting up this test in our tool wouldn’t take more than a few minutes with our Behavioral & Geo-targeting Feature.

4. Remove Your Image Carousels

Image sliders may seem attractive on a homepage, but their reputation in the conversion optimization industry was tainted long ago. Let’s hope you listened and removed those distracting rotating banners. But what next? Check out this post for three ideas you can use to replace your carousels and improve conversions.

5. Give Them Live Chat Assistance

A Forrester Research study found that 44% online consumers want live chat assistance when they are in the middle of a purchase. Practical Commerce reports a case study where live chat boosted conversions by 17% for SupplyGeeks.com. To know more about how you can use live chat better, you can read this awesome article by Kissmetrics.

Product Page Testing Ideas

6. Create Urgency

The time is running out. See how the timer is used by Express Watches on their product page to create urgency:

7. Add Product Videos (At Least for the Popular Products)

Videos are the closest experience users can get online to buy a product. They help people imagine how they will feel when they have the product in their hands. Simply Hike has videos for every product:

According to this report by Reel SEO, visitors who do not watch the video but still have the option to watch them, convert at a higher rate than those who don’t have the option. An article on Smart Insights quotes Matt Lawson, the conversion expert for Appliances Online, where he mentions:

“We have tested and proven that when someone watches our video reviews they’re 120.5% more likely to buy, spend 157.2% longer on the site and spend 9.1% more per order.”

You got the gist. Read my post on how to use videos to boost sales.

8. Price Match Guarantee

See how Webtogs display their guarantee seals and even explains (below the seal in the header) the value of that guarantee to win trust of visitors in an instant.

Bonus points to this site for providing the option to convert prices in different currencies on the product page. Below is the pop up that shows up when you click the option to change country on the product page.

9. Provide 360° View of Products

Larger product images increase conversions. But you probably know that already. Go one step ahead and provide 360 view of your products. Let your customers have no doubts. Give them a closer real life experience with 360° view of products. This will give them the confidence to buy products.
Due Maternity added the 360-spin to their images which increased their conversions by 27%.
Jabong puts this into action pretty well:

10. If You Do Not Provide Free Shipping, Reveal Shipping Fee on the Product Page

We all know charging for shipping is a big conversion killer. But we all have our financial constraints. If you charge for shipping, mention that cost upfront. Ozscopes goes a step ahead and provides a shipping calculator on their product page:

A comScore study revealed that 47% of customers will abandon the cart if they saw that they are being charged for shipping during the checkout.
Sometimes the revenue difference between cheap shipping and free shipping can be huge. There’s no way you can beat the charm of “free shipping.” So if at all you can consider offering free shipping, it can increase your average order value too, in case you haven’t realized it. You can read this article for more information on this.

Checkout Testing Ideas

11. Use Recognizable Security Seals

A survey was conducted to know the security seals that are most effective. The responders were asked which single logo gave them maximum assurance. And these names stood out to be clear winners:

• PayPal (29%)
• Verisign (25%)
• McAfee (23%)
• BBB (7%)
• TRUSTe (3%)

Having security logos that are not recognized or trusted by customers are almost like having no logos at all.

12. Use Reassurance Text

If you do not want to pay for pricey security seals, you can also try adding some reassurance text. Like you can see how Wiltshirefarmfoods.com pulls this off perfectly in the image below (and pass out on any security seals whatsoever):

But one thing you must take note here is, your website’s usability and design should be exceptionally brilliant for this to work.
The text version can sometimes work even better than security seals if your target audience is from an older age group, who do not understand or recognize these security badges.

13. Make Them Generate Promo Codes

All credits to the conversion expert, Craig Sullivan, for this one. The trend is, you ask customers if they have a promo code in the checkout process. And give them a blank box to enter this code if they have it. There are SO many sites who do this to increase sales. But this one can often backfire. Majority of customers leave the site to google for active coupons. Often they get distracted with other stuff they find and never come back to complete their checkout.
Here’s what Craig suggested you can do about it:
Step 1 – Create a link that says “Got a promo code?” instead of a blank box that asks you to enter the promo code.
Step 2 – Once the customer clicks on that link, open a collapsible div that shows two options:
a. Enter the promo code (for those who have it)
b. Generate promo code (for those who do not have it)
You can see this in the image below:

Like it is mentioned in the above image, the promo code generated by customers will only be delivered when they complete their current transaction. This will avoid people leaving the site to find the coupon code if they do not have it. Plus, it will also motivate them to complete their current order, which should reduce your cart abandonment rate.

Thank You Page Testing Ideas

14. Post-Purchase Social Share

Thank you pages are often ignored when people talk about increasing the conversion rate of a website. But not optimizing your thank you page is like asking people to leave from your home (or, website in this case). Sure, they have just bought something and might not be ready to buy something immediately.
But this is where you can tell them what you want them to do. Amazon provides their customers social media options to share the stuff that they have just ordered:

This is great because chances are people in the same network often have similar interests or age group. And if something appealed to one person, it might be liked by other people in their circle too.

15. Ask Them If They Would Like to Participate in a Survey

Knowing your customers always pay off in future. Ask them if they would like to participate in a short survey and get some useful insights from them.
For more ideas on how you can optimize your thank you page, you can read my article here.
Finally, don’t take my word for it. AB test them to see which of these ideas work for you.

Top 9 E-Commerce Usability Guidelines

 



e-Commerce websites are increasing at an incredibly fast rate. There are no boundaries and any geographical restrictions have long disappeared, allowing shopping enthusiasts to shop at their convenience. In spite of these advantages, most e-Commerce websites are performing poorly. e-Commerce websites often fail in their primary objective of making purchasing easy for customers. This article explains common pitfalls and provides a set of e-commerce usability guidelines that can help you make your site more usable.

1. Navigation: Keep it simple. Don’t confuse your customer

Is your navigation structure a complex maze hidden under complicated series of clicks? If yes, this is the biggest strategic blunder in an e-Commerce website. A neat navigation structure will allow smooth and effortless browsing, which ultimately encourages visitors to continue explore the site and turn them into regular site visitors. You need to give tactical thought to the navigation structure of your website to ensure that customers are able to locate information, compare products, and make payment with minimum clicks. To improve the navigation of your e-commerce website, you can head over to this article.

 

2. Homepage: Get it right the first time

Your homepage often creates first impression on the minds of visitors – at least for those who have commenced browsing your site by going through the door. It is imperative that site visitor must find exactly what they’re looking for the moment they arrive. e-Commerce websites need to display numerous product offerings rather than just displaying one or two products on homepage. By displaying very few products on homepage, you are narrowing down customer’s choice from the word go.

3. Product Page Font Visibility: Grab visitor’s attention

Right color combination, font size, and font color ensures that visitors do not have any problem viewing product details and that they find relevant information they are looking for in the shortest possible time. So, every element should be designed to achieve this goal.

4. Add to Cart Button: Final action just a click away

Very often, the ‘Add to Cart’, the final action button in an e-Commerce website is either not well designed or strategically placed to grab prospective buyer’s action. The selection of shape, color, font typography, and button content all trigger the final action. Make sure the ‘Add to Cart’ button is obvious, bright, and prominent in comparison to other features on product page such as wishlists, view product, email to friend, or check out buttons. Less important functions should be lighter colored buttons or simple text links.

5. Font Visibility: A Costly mistake

Another often overlooked e-commerce usability guideline is selecting right typography. Deciding on what font type, font size, and font color to use can be very tricky. You may spend thousands and thousands of dollars on cutting-edge e-Commerce platform and hire most expensive developers for its robust performance, but your site just won’t work if this very basic and crucial aspect of website designing is neglected.


 

6. Primary Navigation: Quick access guaranteed

It is equally important to have an intuitive and engaging primary navigation so that users can find information they are looking for quickly with minimal clicks. For those who are not aware, primary navigation controls are present in a sidebar to the left, right, or top of a page on inner pages. These controls allow users to surf effortlessly on the site. The easier and interesting you make it for your audience, the more likely they are to stay on your site, buy the product they are looking for, and return to your site in future. Dull and mundane primary navigation just won’t work.

7. Trust: Key lies in winning customer’s trust

e-Commerce websites run on the fundamental principle of trust. In order to win customer’s trust it is very important that you study customer psychology. Lack of secure site (https) or lack of a certification by an Internet trust organization can prevent customers from buying from your store. Online businesses must provide security against misuse of confidential information and clearly display privacy policies. Providing guarantees as will client testimonials of actual clients (fake testimonials will hurt you more) will help in improving credibility of your site. It is equally important that you mention details for contacting your organization such as physical address and phone number.

8. Special offers: Don’t save best offers for last

Customers are looking for special offers, discounts, or best deals. Don’t shy away in flaunting such offers at the home page. It is most likely that customer will review special offers and it will strike a chord on the first attempt. Remember the goal of e-Commerce website is not just better user experience, but also to increase sales and improve bottom-line results.


 

9. Breadcrumbs: Where am I?

What is a breadcrumb? It is text-based navigation that shows your current location in the site hierarchy. If your existing e-Commerce site does not have breadcrumbs, it might result in customer discomfort. Customer might feel lost in the maze of product links and rattled if unable to find exact location in site. Highly time-effective, breadcrumbs help in reducing site abandonment and prevent consumer from going off track while placing an order.
Please see an example of breadcrumb below:
Home>>Brands >>Women>>J. Renee
Not only does it give your location within the site, but it also provides shortcuts within the site hierarchy. It is a standard navigational element in most of the robust and successful e-Commerce websites.





 

Conclusion

Success of e-Commerce websites lies in improving user experience, keeping it simple, and winning client’s trust. This will not only result in converting potential clicks into final transaction payment, but also strongly influence the customer to revisit your website in future.