Remove 2017 Remove Data Science Remove Knowledge Discovery Remove Testing
article thumbnail

Explaining black-box models using attribute importance, PDPs, and LIME

Domino Data Lab

After forming the X and y variables, we split the data into training and test sets. Looking at the target vector in the training subset, we notice that our training data is highly imbalanced. All we need to do is instantiate LimeTabularExplainer and give it access to the training data and the independent feature names.

Modeling 139
article thumbnail

ML internals: Synthetic Minority Oversampling (SMOTE) Technique

Domino Data Lab

Their tests are performed using C4.5-generated 1988), E-state data (Hall et al., note that this variant “performs worse than plain under-sampling based on AUC” when tested on the Adult dataset (Dua & Graff, 2017). Data mining for direct marketing: Problems and solutions. Chawla et al., 1998) and others).

article thumbnail

Changing assignment weights with time-based confounders

The Unofficial Google Data Science Blog

Another reason to use ramp-up is to test if a website's infrastructure can handle deploying a new arm to all of its users. The website wants to make sure they have the infrastructure to handle the feature while testing if engagement increases enough to justify the infrastructure. We offer two examples where this may be the case.