article thumbnail

Explaining black-box models using attribute importance, PDPs, and LIME

Domino Data Lab

After forming the X and y variables, we split the data into training and test sets. Looking at the target vector in the training subset, we notice that our training data is highly imbalanced. All we need to do is instantiate LimeTabularExplainer and give it access to the training data and the independent feature names.

Modeling 139
article thumbnail

ML internals: Synthetic Minority Oversampling (SMOTE) Technique

Domino Data Lab

Insufficient training data in the minority class — In domains where data collection is expensive, a dataset containing 10,000 examples is typically considered to be fairly large. Their tests are performed using C4.5-generated 1988), E-state data (Hall et al., Chawla et al., Pima Indian Diabetes (Smith et al.,

article thumbnail

AI, the Power of Knowledge and the Future Ahead: An Interview with Head of Ontotext’s R&I Milena Yankova

Ontotext

Milena Yankova : Our work is focused on helping companies make sense of their own knowledge. Within a large enterprise, there is a huge amount of data accumulated over the years – many decisions have been made and different methods have been tested. Some of this knowledge is locked and the company cannot access it.