article thumbnail

Experiment design and modeling for long-term studies in ads

The Unofficial Google Data Science Blog

A/B testing is used widely in information technology companies to guide product development and improvements. For questions as disparate as website design and UI, prediction algorithms, or user flows within apps, live traffic tests help developers understand what works well for users and the business, and what doesn’t.

article thumbnail

Variance and significance in large-scale online services

The Unofficial Google Data Science Blog

Unlike experimentation in some other areas, LSOS experiments present a surprising challenge to statisticians — even though we operate in the realm of “big data”, the statistical uncertainty in our experiments can be substantial. We must therefore maintain statistical rigor in quantifying experimental uncertainty.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Performing Non-Compartmental Analysis with Julia and Pumas AI

Domino Data Lab

Domino Lab supports both interactive and batch experimentation with all popular IDEs and notebooks (Jupyter, RStudio, SAS, Zeppelin, etc.). Once all packages have been imported, we can move on to loading our test data. We can then proceed with pharmacokinetic modeling, testing the goodness of fit of various models.

Metrics 59
article thumbnail

Changing assignment weights with time-based confounders

The Unofficial Google Data Science Blog

Another reason to use ramp-up is to test if a website's infrastructure can handle deploying a new arm to all of its users. The website wants to make sure they have the infrastructure to handle the feature while testing if engagement increases enough to justify the infrastructure. We offer two examples where this may be the case.