The Unofficial Google Data Science Blog

APRIL 23, 2024

the weight given to Likes in our video recommendation algorithm) while $Y$ is a vector of outcome measures such as different metrics of user experience (e.g., Crucially, it takes into account the uncertainty inherent in our experiments. Figure 2: Spreading measurements out makes estimates of model (slope of line) more accurate.

Experimentation 52

Experimentation Optimization Uncertainty Metrics 52

The Unofficial Google Data Science Blog

DECEMBER 4, 2019

by THOMAS OLAVSON Thomas leads a team at Google called "Operations Data Science" that helps Google scale its infrastructure capacity optimally. This classification is based on the purpose, horizon, update frequency and uncertainty of the forecast. Our team does a lot of forecasting.

Forecasting 113

Forecasting Data Science Statistics Uncertainty 113

O'Reilly on Data

OCTOBER 13, 2020

In Bringing an AI Product to Market , we distinguished the debugging phase of product development from pre-deployment evaluation and testing. During testing and evaluation, application performance is important, but not critical to success. require not only disclosure, but also monitored testing. Debugging AI Products.

Management 244

Management Metrics Machine Learning Modeling 244

The Unofficial Google Data Science Blog

JULY 22, 2020

Another reason to use ramp-up is to test if a website's infrastructure can handle deploying a new arm to all of its users. The website wants to make sure they have the infrastructure to handle the feature while testing if engagement increases enough to justify the infrastructure. We offer two examples where this may be the case.

Experimentation 95

Experimentation Statistics Testing Strategy 95

Domino Data Lab

JUNE 23, 2019

As a result, Skomoroch advocates getting “designers and data scientists, machine learning folks together and using real data and prototyping and testing” as quickly as possible. These measurement-obsessed companies have an advantage when it comes to AI. They have the foundations of data infrastructure. Transcript.

Management 110

Management Machine Learning Experimentation Metrics 110

Data Science and Beyond

JANUARY 7, 2019

Therefore, bootstrapping has been promoted as an easy way of modelling uncertainty to hackers who don’t have much statistical knowledge. Confidence intervals are a common way of quantifying the uncertainty in an estimate of a population parameter. Don’t compare confidence intervals visually.

Statistics 77

Statistics Uncertainty Testing Modeling 77

The Unofficial Google Data Science Blog

MAY 31, 2016

Similarly, we could test the effectiveness of a search ad compared to showing only organic search results. It is important that we can measure the effect of these offline conversions as well. Panel studies make it possible to measure user behavior along with the exposure to ads and other online elements. days or weeks).

Advertising 42

Advertising Testing Sales Statistics 42

Insight

FEBRUARY 19, 2020

Using variability in machine learning predictions as a proxy for risk can help studio executives and producers decide whether or not to green light a film project Photo by Kyle Smith on Unsplash Originally posted on Toward Data Science. I held out 20% of this as a test set and used the remainder for training and validation.

Risk 67

Risk ROI Modeling Metrics 67

The Unofficial Google Data Science Blog

OCTOBER 21, 2015

Our post describes how we arrived at recent changes to design principles for the Google search page, and thus highlights aspects of a data scientist’s role which involve practicing the scientific method. There has been debate as to whether the term “data science” is necessary. Some don’t see the point.

Slice and Dice 45

Slice and Dice Experimentation Data-driven Data Science 45

The Unofficial Google Data Science Blog

APRIL 19, 2021

To explain, let’s borrow a quote from Nate Silver’s The Signal and the Noise : One of the most important tests of a forecast — I would argue that it is the single most important one — is called calibration. The numerical value of the signal became decoupled from the event it was measuring even as the ordinal value remained unchanged.

Modeling 122

Modeling IT Metrics Testing 122

The Unofficial Google Data Science Blog

APRIL 16, 2019

Such decisions involve an actual hypothesis test on specific metrics (e.g. The metrics to measure the impact of the change might not yet be established. Typically, it takes a period of back-and-forth between logging and analysis to gain the confidence that a metric is actually measuring what we designed for it to measure.

Experimentation 68

Experimentation Sales Metrics Measurement 68