The Unofficial Google Data Science Blog

OCTOBER 7, 2015

A/B testing is used widely in information technology companies to guide product development and improvements. For questions as disparate as website design and UI, prediction algorithms, or user flows within apps, live traffic tests help developers understand what works well for users and the business, and what doesn’t.

Modeling 61

Modeling Experimentation Knowledge Discovery Testing 61

The Unofficial Google Data Science Blog

JANUARY 14, 2016

For this purpose, let’s assume we use a t-test for difference between group means. The statistical effect size is often defined as [ e=frac{delta}{sigma} ]which is the difference in group means as a fraction of the (pooled) standard deviation (sometimes referred to as “Cohen’s d” ). known, equal variances). An effect size of 0.2

Experimentation 41

Experimentation Statistics Metrics Measurement 41

The Unofficial Google Data Science Blog

JULY 22, 2020

This post considers a common design for an OCE where a user may be randomly assigned an arm on their first visit during the experiment, with assignment weights referring to the proportion that are randomly assigned to each arm. There are two common reasons assignment weights may change during an OCE.

Experimentation 95

Experimentation Statistics Testing Strategy 95

Domino Data Lab

MAY 20, 2021

Their tests are performed using C4.5-generated 1988), E-state data (Hall et al., note that this variant “performs worse than plain under-sampling based on AUC” when tested on the Adult dataset (Dua & Graff, 2017). References. Data mining for direct marketing: Problems and solutions. Chawla et al.,

Machine Learning 136

Machine Learning Metrics Data mining Knowledge Discovery 136

The Unofficial Google Data Science Blog

NOVEMBER 4, 2015

For more on ad CTR estimation, refer to [2]. One way to check $f_theta$ is to gather test data and check whether the model fits the relationship between training and test data. Figure 4 shows the results of such a test. A machine learning system produces an estimated CTR $t_i$ for each query-ad pair.

KDD 40

KDD Testing Machine Learning Modeling 40

Domino Data Lab

AUGUST 1, 2021

After forming the X and y variables, we split the data into training and test sets. Looking at the target vector in the training subset, we notice that our training data is highly imbalanced. All we need to do is instantiate LimeTabularExplainer and give it access to the training data and the independent feature names.

Modeling 139

Modeling Deep Learning Machine Learning Knowledge Discovery 139