KDD, Measurement and Reference - Data Leaders Brief

Experiment design and modeling for long-term studies in ads

The Unofficial Google Data Science Blog

OCTOBER 7, 2015

by HENNING HOHNHOLD, DEIRDRE O'BRIEN, and DIANE TANG In this post we discuss the challenges in measuring and modeling the long-term effect of ads on user behavior. Nevertheless, A/B testing has challenges and blind spots, such as: the difficulty of identifying suitable metrics that give "works well" a measurable meaning.

Modeling

Modeling Experimentation Knowledge Discovery Testing

Using Empirical Bayes to approximate posteriors for large "black box" estimators

The Unofficial Google Data Science Blog

NOVEMBER 4, 2015

Posteriors are useful to understand the system, measure accuracy, and make better decisions. Methods like the Poisson bootstrap can help us measure the variability of $t$, but don’t give us posteriors either, particularly since good high-dimensional estimators aren’t unbiased. For more on ad CTR estimation, refer to [2].

KDD

KDD Testing Machine Learning Modeling

Data Leaders Brief

Experiment design and modeling for long-term studies in ads

Using Empirical Bayes to approximate posteriors for large "black box" estimators

Webinars

Stay Connected