article thumbnail

Measuring Validity and Reliability of Human Ratings

The Unofficial Google Data Science Blog

Editor's note : The relationship between reliability and validity are somewhat analogous to that between the notions of statistical uncertainty and representational uncertainty introduced in an earlier post. If they roll two dice and apply a label if the dice rolls sum to 12 they will agree 85% of the time, purely by chance.

article thumbnail

Data scientist as scientist

The Unofficial Google Data Science Blog

Note also that this account does not involve ambiguity due to statistical uncertainty. As you can see from the tiny confidence intervals on the graphs, big data ensured that measurements, even in the finest slices, were precise. We sliced and diced the experimental data in many many ways.