Why Your A/B Tests Are Failing
In a 2013 study by eConsultancy & RedEye, surveying almost 1,000 client-side & agency marketers, it was found that 60% found A/B testing to be “quite valuable” to their business.
Yet, only just over a quarter (28%) report being satisfied with their conversion rates.
What’s interesting is that another study by VWO found that only 1 in every 7 A/B tests is a statistically significant winning test. In our own research at Convert.com, we analysed 700 experiments and found again that 1 out of 7 experiments (14%) ran made a positive impact on the conversion rate for all clients that were not an agency.
But here’s where it get’s interesting… for the clients who did use an agency that specialized in conversion optimization, 1 out of every 3 (33%) tests drove a statistically significant result.
Yes, you are reading this right: many conversion optimization agencies get almost triple chances on a positive A/B test output vs. other clients using the same A/B testing tool.