Experimental Design and Statistical Inference
-
A very illustrative example of a Bayesian approach to A/B testing from the book Bayesian Methods for hackers.
-
A critically important aspect of any study is determining the appropriate sample size to answer the research question. This module will focus on formulas that can be used to estimate the sample size needed to produce a confidence interval estimate with a specified margin of error (precision) or to ensure that a test of hypothesis has a high probability of detecting a meaningful difference in the parameter.
-
Innovating Faster on Personalization Algorithms at Netflix Using Interleaving
To accelerate the pace of algorithm innovation, we have devised a two-stage online experimentation process. The first stage is a fast pruning step in which we identify the most promising ranking algorithms from a large initial set of ideas. The second stage is a traditional A/B test on the pared-down set of algorithms to measure their impact on longer-term member behavior. In this blog post, we focus on our approach to the first stage: an interleaving technique that unlocks our ability to more precisely measure member preferences.
-
A Crash Course in Causality: Inferring Causal Effects from Observational Data, Jason Roy
We have all heard the phrase “correlation does not equal causation.” What, then, does equal causation? This course aims to answer that question and more!