Controlled Experimentation (or A/B testing) has evolved into a powerful tool for driving product strategy and innovation. The dramatic growth in online and mobile content, media, and commerce has enabled companies to make principled data-driven decisions. Large numbers of experiments are typically run to validate hypotheses, study causation, and optimize user experience, engagement, and monetization.

The popularity of controlled experimentation has given birth to several companies that specialize in offering tools and services to effectively setup, manage, and monitor large-scale tests. While the simplicity of this concept is disarming, there are several pitfalls that can stymie progress and occasionally lead to poor decisions.

In this talk from The Hive’s Big Data Think Tank Meetup at Microsoft Rajesh Parekh discusses the guiding of product innovation with controlled experimentation:

Podcast: Play in new window | Download