article thumbnail

Measuring Validity and Reliability of Human Ratings

The Unofficial Google Data Science Blog

Editor's note : The relationship between reliability and validity are somewhat analogous to that between the notions of statistical uncertainty and representational uncertainty introduced in an earlier post. If they roll two dice and apply a label if the dice rolls sum to 12 they will agree 85% of the time, purely by chance.

article thumbnail

Data scientist as scientist

The Unofficial Google Data Science Blog

The beliefs of this community are always evolving, and the process of thoughtfully generating, testing, refuting and accepting ideas looks a lot like Science. Note also that this account does not involve ambiguity due to statistical uncertainty. We sliced and diced the experimental data in many many ways.