article thumbnail

Changing assignment weights with time-based confounders

The Unofficial Google Data Science Blog

For example, consider a smaller website that is considering adding a video hosting feature to increase engagement on the site. The fantasy football and video hosting examples, which we will discuss in more detail later, highlight situations where this design might be considered, despite potential complexity in the analysis.

article thumbnail

Measuring Validity and Reliability of Human Ratings

The Unofficial Google Data Science Blog

Editor's note : The relationship between reliability and validity are somewhat analogous to that between the notions of statistical uncertainty and representational uncertainty introduced in an earlier post. But for more complicated metrics like xRR, our preference is to bootstrap when measuring uncertainty.