The Unofficial Google Data Science Blog

OCTOBER 7, 2015

by HENNING HOHNHOLD, DEIRDRE O'BRIEN, and DIANE TANG In this post we discuss the challenges in measuring and modeling the long-term effect of ads on user behavior. We describe experiment designs which have proven effective for us and discuss the subtleties of trying to generalize the results via modeling.

Modeling 61

Modeling Experimentation Knowledge Discovery Testing 61

The Unofficial Google Data Science Blog

JANUARY 14, 2016

But the fact that a service could have millions of users and billions of interactions gives rise to both big data and methods which are effective with big data. Of particular interest to LSOS data scientists are modeling and prediction techniques which keep improving with more data.

Experimentation 41

Experimentation Statistics Metrics Measurement 41

The Unofficial Google Data Science Blog

JULY 22, 2020

For this reason we don’t report uncertainty measures or statistical significance in the results of the simulation. Ramp-up solution: measure epoch and condition on its effect If one wants to do full traffic ramp-up and use data from all epochs, they must use an adjusted estimator to get an unbiased estimate of the average reward in each arm.

Experimentation 95

Experimentation Statistics Testing Strategy 95

Domino Data Lab

MAY 20, 2021

In this article we discuss why fitting models on imbalanced datasets is problematic, and how class imbalance is typically addressed. This renders measures like classification accuracy meaningless. The use of multiple measurements in taxonomic problems. The unreasonable effectiveness of data. Machine Learning, 57–78.

Machine Learning 136

Machine Learning Metrics Data mining Knowledge Discovery 136

The Unofficial Google Data Science Blog

NOVEMBER 4, 2015

Posteriors are useful to understand the system, measure accuracy, and make better decisions. But most common machine learning methods don’t give posteriors, and many don’t have explicit probability models. In our model, $theta$ doesn’t depend directly on $x$ — all the information in $x$ is captured in $t$.

KDD 40

KDD Testing Machine Learning Modeling 40

The Unofficial Google Data Science Blog

FEBRUARY 29, 2016

And since the metric average is different in each hour of day, this is a source of variation in measuring the experimental effect. Rare binary event example In the previous post , we discussed how rare binary events can be fundamental to the LSOS business model. Let $Y_i$ be the response measured on the $i$th user session.

Experimentation 40

Experimentation Metrics Statistics Measurement 40

Domino Data Lab

AUGUST 1, 2021

In this article we cover explainability for black-box models and show how to use different methods from the Skater framework to provide insights into the inner workings of a simple credit scoring neural network model. The interest in interpretation of machine learning has been rapidly accelerating in the last decade. See Ribeiro et al.

Modeling 139

Modeling Deep Learning Machine Learning Knowledge Discovery 139