2011, Data Quality, Measurement and Uncertainty

Measuring Validity and Reliability of Human Ratings

The Unofficial Google Data Science Blog

JULY 18, 2023

E ven after we account for disagreement, human ratings may not measure exactly what we want to measure. How do we think about the quality of human ratings, and how do we quantify our understanding is the subject of this post. While human-labeled data is critical to many important applications, it also brings many challenges.

Measurement

Measurement Metrics Uncertainty Slice and Dice

Fact-based Decision-making

Peter James Thomas

AUGUST 12, 2018

This piece was prompted by both Olaf’s question and a recent article by my friend Neil Raden on his Silicon Angle blog, Performance management: Can you really manage what you measure? These and other areas are covered in greater detail in an older article, Using BI to drive improvements in data quality.

Metrics

Metrics Statistics Data Quality Measurement

Data Leaders Brief

Measuring Validity and Reliability of Human Ratings

Fact-based Decision-making

Webinars

Stay Connected