Remove 2011 Remove Data Collection Remove Measurement Remove Uncertainty
article thumbnail

Measuring Validity and Reliability of Human Ratings

The Unofficial Google Data Science Blog

E ven after we account for disagreement, human ratings may not measure exactly what we want to measure. Overview Human-labeled data is ubiquitous in business and science, and platforms for obtaining data from people have become increasingly common. And for thousands of years, measurement was as simple as this.

article thumbnail

The Lean Analytics Cycle: Metrics > Hypothesis > Experiment > Act

Occam's Razor

We are far too enamored with data collection and reporting the standard metrics we love because others love them because someone else said they were nice so many years ago. First, you figure out what you want to improve; then you create an experiment; then you run the experiment; then you measure the results and decide what to do.

Metrics 156
article thumbnail

Our quest for robust time series forecasting at scale

The Unofficial Google Data Science Blog

Quantification of forecast uncertainty via simulation-based prediction intervals. We conclude with an example of our forecasting routine applied to publicly available Turkish Electricity data. They can arise from data collection errors or other unlikely-to-repeat causes such as an outage somewhere on the Internet.