The Unofficial Google Data Science Blog

JULY 18, 2023

E ven after we account for disagreement, human ratings may not measure exactly what we want to measure. Researchers and practitioners have been using human-labeled data for many years, trying to understand all sorts of abstract concepts that we could not measure otherwise. That’s the focus of this blog post.

Measurement 52

Measurement Metrics Uncertainty Slice and Dice 52

The Unofficial Google Data Science Blog

APRIL 23, 2024

the weight given to Likes in our video recommendation algorithm) while $Y$ is a vector of outcome measures such as different metrics of user experience (e.g., Crucially, it takes into account the uncertainty inherent in our experiments. Figure 2: Spreading measurements out makes estimates of model (slope of line) more accurate.

Experimentation 52

Experimentation Optimization Uncertainty Metrics 52

Peter James Thomas

AUGUST 12, 2018

This piece was prompted by both Olaf’s question and a recent article by my friend Neil Raden on his Silicon Angle blog, Performance management: Can you really manage what you measure? It is hard to account for such tweaking in measurement systems. Some relate to inherent issues with what is being measured. million.

Metrics 49

Metrics Statistics Data Quality Measurement 49

The Unofficial Google Data Science Blog

JULY 22, 2020

For example, imagine a fantasy football site is considering displaying advanced player statistics. A ramp-up strategy may mitigate the risk of upsetting the site’s loyal users who perhaps have strong preferences for the current statistics that are shown. One reason to do ramp-up is to mitigate the risk of never before seen arms.

Experimentation 95

Experimentation Statistics Testing Strategy 95

The Unofficial Google Data Science Blog

APRIL 17, 2017

Quantification of forecast uncertainty via simulation-based prediction intervals. First, the system may not be understood, and even if it was understood it may be extremely difficult to measure the relationships that are assumed to govern its behavior. Crucially, our approach does not rely on model performance on holdout samples.

Forecasting 83

Forecasting Modeling Statistics Uncertainty 83

The Unofficial Google Data Science Blog

MARCH 13, 2017

Typically, causal inference in data science is framed in probabilistic terms, where there is statistical uncertainty in the outcomes as well as model uncertainty about the true causal mechanism connecting inputs and outputs. 2009, " Measuring invariances in deep networks ". CoRR, 2016. [3] Goodfellow, Quoc V.

IT 68

IT Visualization Modeling Uncertainty 68

O'Reilly on Data

DECEMBER 9, 2019

Because of this trifecta of errors, we need dynamic models that quantify the uncertainty inherent in our financial estimates and predictions. Practitioners in all social sciences, especially financial economics, use confidence intervals to quantify the uncertainty in their estimates and predictions.

Statistics 27

Statistics Uncertainty Risk Marketing 27