Remove 2009 Remove Data mining Remove Risk Remove Testing
article thumbnail

ML internals: Synthetic Minority Oversampling (SMOTE) Technique

Domino Data Lab

The problem with this approach is that in highly imbalanced sets it can easily lead to a situation where most of the data has to be discarded, and it has been firmly established that when it comes to machine learning data should not be easily thrown out (Banko and Brill, 2001; Halevy et al., Their tests are performed using C4.5-generated

article thumbnail

Explaining black-box models using attribute importance, PDPs, and LIME

Domino Data Lab

For this demo we’ll use the freely available Statlog (German Credit Data) Data Set, which can be downloaded from Kaggle. This dataset classifies customers based on a set of attributes into two credit risk groups – good or bad. After forming the X and y variables, we split the data into training and test sets.

Modeling 139
article thumbnail

Misleading Statistics Examples – Discover The Potential For Misuse of Statistics & Data In The Digital Age

datapine

To make sure the reliability is high, there are various techniques to perform – the first of them being the control tests, which should have similar results when reproducing an experiment in similar conditions. A 2009 investigative survey by Dr. Daniele Fanelli from The University of Edinburgh found that 33.7% 3) Data fishing.