Multivariate Testing

Multivariate testing allows the testing of different versions of a webpage to accurately identify the most effective one. Visitor statistics and interactions are measured to show the difference in performance. Multivariate testing looks at “elemental contribution”; the elements that contribute to an increase/decrease in conversion. These elements range from the colour of a link to the length of a form on a page.

A tailored process
This will help you to understand the “triggers” that influence conversion. It is particularly important to remember that the changes that work on some websites will not work on others. If by changing the colour of a submit button from green to red improves conversion, the same change won’t necessarily work on another site. Multivariate testing is a process tailored to your own site and users.

How to do multivariate testing

1. Select the tool you want to use to test (A list of software is available at the end of this post)
2. Design test (Test bounce rate on the homepage? Or conversion rate on the payment area?)
3. Develop creative (What elements will you change? The font? The text on the page?)
4. Implement code (This activates the Multivariate testing and will send users to different pages)
5. Monitor and learn
6. Repeat (Look at other elements that can be changed or improved)

The benefits of Multivariate Testing
Multivariate testing can be used to justify design and layout changes. Instead of relying on the “gut feel” of your designers, the test results will make the decision for you. You can use MV results to put some support behind any design ideas you suggest.

You can remove any subjective arguments and opinions about the design and layout of your site and use multivariate testing results to justify changes. This is very useful when working client side in large corporate environments. It can show loss in conversion/revenue which is an easy way to get support from senior managers.

When Multivariate testing remember…

• You are testing with real people
• Unlike A/B testing, Multivariate testing allows you more potential combinations
• You need to collect a substantial amount of data to support a decision to change a webpage permanently. Decide how substantial the amount needs to be beforehand
• Good test results are sometimes unexplainable and might even go against common usability thinking or advice
• To combine testing with other technology like gaze trails and heatmap imaging to help decide what elements to alter and test
• To test pages that will make a difference, test key landing pages
• Make things simple and easy for users (See Paradox of choice below)

The Paradox of choice
Faced with two much choice a consumer will often choose not to purchase. This is because the increased cognitive thought needed to make a decision will burden the user too much. This is a key thing to remember when choosing what element to alter when Multivariate testing. Keep calls to action short and simple. Often text on a button performs better when kept short: “Submit” would usually perform better than “Submit your email details”. This is just because a shorter piece of text requires less brain power to interpret, and act on.

Multivariate Testing software

• Google Optimiser
• Maxymiser
• Optimost
• Omniture test and target

Any success stories using multivariate testing? Let me know your thoughts, experiences and comments!