Dr. Christian Posse was the last panelist at the recent Controlled Experimentation (A/B Testing) Meetup at Microsoft. In this talk, Christian shares some of the problems he’s seen in the social network field. Not a single piece of code, algorithm, feature, or user experience goes out without A/B Testing. He discusses their development of a system of hashing functions over at LinkedIn that allow them to run millions of A/B tests concurrently without interactions between them.
Podcast: Play in new window | Download
Slides & Bio
Original post with audio and slides is here.
Hey guys, you’re listening to g33ktalk. Today, we’ve got a talk by Ted Dunning from the Hadoop meet up in New York last night. Ted is going to talk to us about recent developments in Mahout. Stay tuned.
Ted Dunning: Okay, what I want to do is talk about news from Mahout, as if Mahout were a foreign land. Sometimes I think it is. I am chief application architect for MapR. Cool hat, huh? These hats are on offer. They go to employees— at least the black belt version goes to employee. And so, if you think you qualify for the black belt version, get in touch with us. We’re definitely hiring. You can get in touch with me a lot of ways. First initial, last name at maprtech. First initial, last name at apache dot org. Or, , ted_dunning @ Twitter. Any way you like. I try to answer everything, if anybody sends me anything, which sometimes means I work too late.
Continue reading »
This talk is the fourth of five from The Hive’s Big Data Think Tank Meetup at Microsoft. Ya Xu discusses controlled experimentation at Microsoft’s Bing in her talk entitled “Why Didn’t My Feature Improve the Metric?”:
Podcast: Play in new window | Download
Continue reading »
Controlled Experimentation (or A/B testing) has evolved into a powerful tool for driving product strategy and innovation. The dramatic growth in online and mobile content, media, and commerce has enabled companies to make principled data-driven decisions. Large numbers of experiments are typically run to validate hypotheses, study causation, and optimize user experience, engagement, and monetization.
The concept of controlled experimentation is simple – randomly divide the user population into two groups called the Control (A) and Treatment (B). The experiment involves concurrently exposing the Control group users to one experience (typically, the existing experience) and the Treatment group users to another (the new experience). A set of performance metrics are computed for both groups and statistical tests are run to determine if the change in metrics (if any) for the Treatment group compared to the Control group is purely due to chance or not. Typical use cases of A/B testing include, testing a modified web user interface, evaluating a new call to action for mobile app downloads, and examining the effects of a new personalization algorithm.
In this talk, the second of 5 recorded from The Hive’s Big Data Think Tank Meetup at Microsoft, discusses controlled experimentation as they’ve seen it at Netflix.
Podcast: Play in new window | Download
Continue reading »