article thumbnail

Towards optimal experimentation in online systems

The Unofficial Google Data Science Blog

the weight given to Likes in our video recommendation algorithm) while $Y$ is a vector of outcome measures such as different metrics of user experience (e.g., Taking measurements at parameter settings further from control parameter settings leads to a lower variance estimate of the slope of the line relating the metric to the parameter.

article thumbnail

Drug Discovery Needs AI To Discover More Treatments

Smart Data Collective

In a report on the failure rates of drug discovery efforts between 2013 and 2015, Richard K. An open and impartial AI model should be able to inject a measure of transparency into this process along with the obvious efficiency advantages. Unfortunately, a substantial number of clinical trials fails in these two Phases.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Experiment design and modeling for long-term studies in ads

The Unofficial Google Data Science Blog

by HENNING HOHNHOLD, DEIRDRE O'BRIEN, and DIANE TANG In this post we discuss the challenges in measuring and modeling the long-term effect of ads on user behavior. Nevertheless, A/B testing has challenges and blind spots, such as: the difficulty of identifying suitable metrics that give "works well" a measurable meaning.

article thumbnail

Changing assignment weights with time-based confounders

The Unofficial Google Data Science Blog

Instead, we focus on the case where an experimenter has decided to run a full traffic ramp-up experiment and wants to use the data from all of the epochs in the analysis. When there are changing assignment weights and time-based confounders, this complication must be considered either in the analysis or the experimental design.

article thumbnail

The 2015 Digital Marketing Rule Book. Change or Perish.

Occam's Razor

Most companies are astonishingly blasé about data and possibilities of measurement. " Sad, unimaginative measurements of their sad, unimaginative campaigns. Allocate some of your aforementioned 15% budget to experimentation and testing. This blog is about the joys of measurement and the transformative power of data.

Marketing 140
article thumbnail

To Balance or Not to Balance?

The Unofficial Google Data Science Blog

In an ideal world, experimentation through randomization of the treatment assignment allows the identification and consistent estimation of causal effects. See Hainmueller (2012), and the work of Zhao & Percival (2015) for more details on how this optimization problem is solved, and for further discussion. R package version 2.0-15.

article thumbnail

Themes and Conferences per Pacoid, Episode 6

Domino Data Lab

Spoiler alert: a research field called curiosity-driven learning is emerging at the nexis of experimental cognitive psychology and industry use cases for machine learning, particularly in gaming AI. The ability to measure results (risk-reducing evidence). Ensure a culture that supports a steady process of learning and experimentation.