Pulling Back The Curtain on P-Values (or How I Learned To Love Small Data)


For all of the talk about how awesome (and big, don’t forget big) Big data is, one of the favorite tools in the conversion optimization toolkit, A/B Testing, is decidedly small data.

Optimization, winners and losers, Lean this that or the other thing, at the end of the day, A/B Testing is really just an application of sampling.

You take couple of alternative options (eg. ‘50% off’ v ‘Buy One Get One Free’ ) and try them out with a portion of your users. You see how well each one did and then make a decision about which one you think will give you the most return.

Sounds simple, and in a way it is, yet there seem to be lots of questions around, significance testing, and in particular what the heck is the p-value is, and how to interpret it to help best make sound business decisions.

These are actually deep questions & in order to begin to get a handle on them, you need to have a basic grasp of sampling.

Hey, Cut The Curd!

I had a math professor in college who would go absolutely ballistic whenever anyone in class made a basic conceptual mistake. He must have grown up on …read more

Article Curated From…: Conversion XL