The Unofficial Google Data Science Blog

JANUARY 16, 2018

Experimentation on networks A/B testing is a standard method of measuring the effect of changes by randomizing samples into different treatment groups. However, this assumption no longer holds when samples interact with each other, such as in a network. Consider the case where experiment metrics are evaluated at the per-user level.

Testing 58

Testing Experimentation Measurement Modeling 58

Business Over Broadway

MAY 22, 2019

According to Gartner, companies need to adopt these practices: build culture of collaboration and experimentation; start with a 3-way partnership among executives leading digital initiative, line of business and IT. Also, loyalty leaders infuse analytics into CX programs, including machine learning, data science and data integration.

Machine Learning 40

Machine Learning Technology Digital Transformation Data Science 40

Insight

MARCH 12, 2020

The most powerful approach for the first task is to use a ‘language model’ (LM), i.e. a statistical model of natural language. After some experimentation, I landed on a strategy I’ll call ‘warm encoding’: if greater than 1% of tags were in a particular class, I encoded the book as belonging to that class, non-exclusively.

Modeling 59

Modeling Metadata Publishing Sales 59

Occam's Razor

JUNE 6, 2007

Bonus: Interactive CD: Contains six podcasts, one video, two web analytics metrics definitions documents and five insightful powerpoint presentations. Experimentation & Testing (A/B, Multivariate, you name it). Bonus: Interactive CD. Immediately actionable web analytics (your biggest worries covered).

Analytics 60

Analytics Metrics Marketing Dashboards 60

Domino Data Lab

JUNE 23, 2019

Skomoroch proposes that managing ML projects are challenging for organizations because shipping ML projects requires an experimental culture that fundamentally changes how many companies approach building and shipping software. Another pattern that I’ve seen in good PMs is that they’re very metric-driven.

Management 110

Management Machine Learning Experimentation Metrics 110

Corinium

APRIL 25, 2019

Life insurance needs accurate data on consumer health, age and other metrics of risk. For example auto insurance companies offering to capture real-time driving statistics from policy-holders’ cars to encourage and reward safe driving. And more recently, we have also seen innovation with IOT (Internet Of Things).

Insurance 150

Insurance Risk IoT Cost-Benefit 150

The Unofficial Google Data Science Blog

APRIL 16, 2019

by MICHAEL FORTE Large-scale live experimentation is a big part of online product development. This means a small and growing product has to use experimentation differently and very carefully. This blog post is about experimentation in this regime. Such decisions involve an actual hypothesis test on specific metrics (e.g.

Experimentation 68

Experimentation Sales Metrics Measurement 68

The Unofficial Google Data Science Blog

JULY 31, 2016

To figure this out, let's consider an appropriate experimental design. In other words, the teacher is our second kind of unit, the unit of experimentation. This type of experimental design is known as a group-randomized or cluster-randomized trial. The second source of dependence comes from student interactions.

Experimentation 42

Experimentation Testing Measurement Metrics 42

The Unofficial Google Data Science Blog

MARCH 22, 2018

But what if users don't immediately uptake the new experimental version? Background At Google, experimentation is an invaluable tool for making decisions and inference about new products and features. by DANIEL PERCIVAL Randomized experiments are invaluable in making product decisions, including on mobile apps.

Experimentation 62

Experimentation Measurement Modeling Statistics 62

Occam's Razor

MARCH 23, 2010

Part of it is fueled by a vocal minority genuinely upset that 10 years on we are still not a statistically powered bunch doing complicated analysis that is shifting paradigms. Because every tool uses its own sweet metrics definitions, cookie rules, session start and end rules and so much more. Part of it fueled by some Consultants.

Analytics 118

Analytics Consulting Metrics Experimentation 118

The Unofficial Google Data Science Blog

OCTOBER 21, 2015

It is important to make clear distinctions among each of these, and to advance the state of knowledge through concerted observation, modeling and experimentation. These quasi-explanations usually involve large, real effects and interactions so complex that arguments based on them are often non-falsifiable.

Slice and Dice 45

Slice and Dice Experimentation Data-driven Data Science 45

Domino Data Lab

MAY 8, 2019

If your “performance” metrics are focused on predictive power, then you’ll probably end up with more complex models, and consequently less interpretable ones. They also require advanced skills in statistics, experimental design, causal inference, and so on – more than most data science teams will have.

Machine Learning 105

Machine Learning Data Science Modeling Visualization 105

Domino Data Lab

AUGUST 22, 2019

Although it’s not perfect, [Note: These are statistical approximations, of course!] Instead, we recommend using the bokeh library to create a highly interactive—and actionable—plot, as with the code provided in Example 11.11. Interactive bokeh plot of two-dimensional word-vector data. Example 11.6 Example 11.11 y=subset_df.y,

Deep Learning 111

Deep Learning Modeling Metrics Testing 111

Domino Data Lab

AUGUST 26, 2019

To support the iterative and experimental nature of industry work, Domino reached out to Addison-Wesley Professional (AWP) for appropriate permissions to excerpt the “Tuning Hyperparameters and Pipelines” from the book, Machine Learning with Python for Everyone by Mark E. algorithm leaf_size metric metric_params n_jobs n_neighbors p weights.

Testing 79

Testing Modeling Machine Learning Metrics 79

IBM Big Data Hub

SEPTEMBER 25, 2023

Presto is an open source distributed SQL query engine for data analytics and the data lakehouse, designed for running interactive analytic queries against datasets of all sizes, from gigabytes to petabytes. Data Exploration and Innovation: The flexibility of Presto has encouraged data exploration and experimentation at Uber.

OLAP 92

OLAP Data Lake Data-driven Snapshot 92