The Unofficial Google Data Science Blog

JULY 22, 2020

One reason to do ramp-up is to mitigate the risk of never before seen arms. A ramp-up strategy may mitigate the risk of upsetting the site’s loyal users who perhaps have strong preferences for the current statistics that are shown. For example, imagine a fantasy football site is considering displaying advanced player statistics.

Experimentation 95

Experimentation Statistics Testing Strategy 95

Domino Data Lab

MAY 20, 2021

This renders measures like classification accuracy meaningless. This carries the risk of this modification performing worse than simpler approaches like majority under-sampling. note that this variant “performs worse than plain under-sampling based on AUC” when tested on the Adult dataset (Dua & Graff, 2017). Chawla et al.

Machine Learning 136

Machine Learning Metrics Data mining Knowledge Discovery 136

Domino Data Lab

AUGUST 1, 2021

This dataset classifies customers based on a set of attributes into two credit risk groups – good or bad. This is to be expected, as there is no reason for a perfect 50:50 separation of the good vs. bad credit risk. but it generally relies on measuring the entropy in the change of predictions given a perturbation of a feature.

Modeling 139

Modeling Deep Learning Machine Learning Knowledge Discovery 139