article thumbnail

Fundamentals of Data Mining

Data Science 101

This data alone does not make any sense unless it’s identified to be related in some pattern. Data mining is the process of discovering these patterns among the data and is therefore also known as Knowledge Discovery from Data (KDD). You might be wondering what benefit you can get out of these techniques?

article thumbnail

Density-Based Clustering

Domino Data Lab

Due to its importance in both theory and applications, this algorithm is one of three algorithms awarded the Test of Time Award at the KDD conference in 2014. Rather it infers the number of clusters based on the data, and it can discover clusters of arbitrary shape (for comparison, k-means usually discovers spherical clusters).

Metrics 116
article thumbnail

Using Empirical Bayes to approximate posteriors for large "black box" estimators

The Unofficial Google Data Science Blog

Empirical Bayes methods find a prior such that when we add Poisson noise, we fit the distribution of our observed data. For an introduction to Empirical Bayes, see the paper [3] by Brad Efron (with more in his book [4]). In Figure 2, the red line shows a Gamma prior that leads to a good fit. How exactly should we model $G$?

KDD 40