Experimentation and Testing: A Primer

Occam's Razor

This post is a primer on the delightful world of testing and experimentation (A/B, Multivariate, and a new term from me: Experience Testing). It is important to realize that experimentation and testing might sound big and complex but it is not.

Six Nudges: Creating A Sense Of Urgency For Higher Conversion Rates!

Occam's Razor

Marketing Tips Usability Voice of Customer conversion rate experimentation and testing user experienceBy every indicator available, ecommerce is continuing to grow at an insane speed.

Eight Silly Data Things Marketing People Believe That Get Them Fired.

Occam's Razor

competitive intelligence Digital Analytics Digital Marketing Marketing Tips Search Engine Marketing Social Media Web Metrics actionable web analytics digital marketing experimentation and testing marketing metrics

The Lean Analytics Cycle: Metrics > Hypothesis > Experiment > Act

Occam's Razor

Advanced Analytics Big Data Digital Analytics Web Analytics Web Insights Web Metrics actionable analytics business optimization experimentation and testing key performance indicators This should not be news to you.

Driving Discovery and Experimentation in your Organization

Speaker: Teresa Torres, Product Discovery Coach, Product Talk, David Bland, Founder and CEO, Precoil, and Hope Gurion, Product Coach and Advisor, Fearless Product LLC

Measuring Incrementality: Controlled Experiments to the Rescue!

Occam's Razor

Having read this post what might be the biggest downside to experimentation? Advanced Analytics Marketing Tips Search Engine Marketing acquisition portfolio optimization actionable web analytics excellent analytics tips experimentation and testing

Industrializing your AI and data science models with IBM Cloud Private for Data

IBM Big Data Hub

The next chapter is all about moving from experimentation to true transformation. Companies are entering “chapter two” of their digital transformation. It’s about gaining speed and scale.

Machine Learning Product Management: Lessons Learned

Domino Data Lab

Pete indicates, in both his November 2018 and Strata London talks, that ML requires a more experimental approach than traditional software engineering. These steps also reflect the experimental nature of ML product management.

How Analytics by Design Tackles The Yin and Yang of Metrics and Data

Kirk Borne

By applying an agile methodology, we are able to adopt a culture of experimentation that permits us to fail fast in order to learn fast and that delivers both the minimum viable product and the minimum lovable product. Written by Dr. Kirk Borne.

Interview with: Sankar Narayanan, Chief Practice Officer at Fractal Analytics


It is also important to have a strong test and learn culture to encourage rapid experimentation. Will you please describe your role at Fractal Analytics? I am the Chief Practice Officer for Insurance, Healthcare, and Hi-Tech verticals at Fractal.

Managing Risk in Data Projects


It’s probably safe to say that for at least some of those explorers, the prospect of risk when it comes to data and AI projects is paralyzing, causing them to stay in a phase of experimentation.

Hey Siri, What’s My Forecasted EBITDA Look Like?


Experimental” Technology. Is AI truly experimental technology? Even though we have so much advanced technology surrounding us, we still cannot just ask, “ Hey Siri, what’s my forecasted EBITDA look like ?” There are many reasons why such technology isn’t available yet—insufficient data, unstructured data and some human knowledge that is not yet transferable to machine.

Announcing Domino 3.3: Datasets and Experiment Manager

Domino Data Lab

Data science is different from other workstreams like software development in that it involves open-ended exploration and experimentation to find optimal solutions. Our mission at Domino is to enable organizations to put models at the heart of their business.

Is Google Cloud Platform Ready to Run Your Data Analytics Pipeline?

Sanjeev Mohan

GCP has gained acceptance for development and experimentation and more enterprise customers are putting it into production. Is Google Cloud Platform Ready to Run Your Data Analytics Pipeline? This is the focus of my latest research which published in Jan 2019.

Ask Why! Finding motives, causes, and purpose in data science

Data Science and Beyond

Causality and experimentation. Some people equate predictive modelling with data science, thinking that mastering various machine learning techniques is the key that unlocks the mysteries of the field. However, there is much more to data science than the What and How of predictive modelling.

Open Data Science and Machine Learning for Business with Cloudera Data Science Workbench on HDP


Data scientists require on-demand access to data, powerful processing infrastructure, and multiple tools and libraries for development and experimentation.

The Role of Theory in Data Analysis

Simply Statistics

Many data analysts are not involved in the data collection process or the experimental design and so it is important to inquire about he process by which the data came to them. But when she visited the lab one day to see how the experiments were done, she discovered that the experimental units were all processed in one batch and the control units were all processed in a different batch at a different time, thereby confounding any treatment effect with the batch.

Misadventures in experiments for growth

The Unofficial Google Data Science Blog

by MICHAEL FORTE Large-scale live experimentation is a big part of online product development. This means a small and growing product has to use experimentation differently and very carefully. This blog post is about experimentation in this regime.

Further Exploration #13: 3D Treemap Spheres and Cylinders

The Data Visualisation Catalogue

This experimental visualisation uses a polar layout and extends into into 3D with a cylindrical form. Another experimental variation of a Treemap was carried out by Schulz and co, by taking the 3D Polar Treemap a step further by combining it with elements of the Steptree.

Glossary of Digital Terminology for Career Relevance

Rocket-Powered Data Science

Computer Vision: Data Mining: Data Science: Application of scientific method to discovery from data (including Statistics, Machine Learning, data visualization, exploratory data analysis, experimentation, and more). Career Relevance. Definitions of terminology frequently seen and used in discussions of emerging digital technologies. NOTE: This page is a WIP = Work In Progress.).

Comparing the Functionality of Open Source Natural Language Processing Libraries

Domino Data Lab

If you are in research, excellent libraries like Allen NLP and NLP Architect are designed to make experimentation easier, although at the expense of feature completeness, speed and robustness.

Keynote Takeaways From Gartner Data & Analytics Summit


Gartner chose to group the rest of the keynote into three main messages according to the following categories: Here are some of the highlights as presented for each of them: Data Driven – “Adopt an Experimental Mindset”.

AI in Analytics: The NLQ Use Case


When the app is first opened, the user may be searching for a specific song that was heard while passing by the neighborhood cafe, or the user may want to be surprised with, let’s say, a song from the new experimental album by a Yemen Reggae folk artist.

Worthy of Your Attention

Perceptual Edge

Stigchel is a professor in the Department of Experimental Psychology at Utrecht University in the Netherlands. I spend a great deal of time reading books.

Augmented Data Discovery Provides Users with Crucial Answers


Advanced Data Discovery ensures data democratization and can drastically reduce the time and cost of analysis and experimentation. Advanced Data Discovery Can and Should Be Available to All!

Interview with Dominic Sartorio, Senior Vice President for Products & Development, Protegrity


Whether eventual legislation will exactly mirror GDPR remains to be seen, I think there will be some experimentation at the State level as well as for specific verticals whose successes would point the way.

Predictive Analytics Can Guide the Organization to Success


Plug n’ Play Predictive Analysis for Accurate Forecasting! There are numerous considerations when a business looks at upgrading or acquiring an analytical solution. One very important capability is Put n’ Play predictive analysis.

Machine Learning Integration Options

Paul DeBeasi

Machine learning projects are inherently different from traditional IT projects in that they are significantly more heuristic and experimental, requiring skills spanning multiple domains, including statistical analysis, data analysis and application development.

The ethics of data flow

O'Reilly on Data

Principles for ethical data handling (and human experimentation in general) always stress "informed consent"; Nissenbaum’s discussion about context suggests that informed consent is less about usage than about data flow.

The most practical causal inference book I’ve read (is still a draft)

Data Science and Beyond

The book focuses on randomised controlled trials and well-defined interventions as the basis of causal inference from both experimental and observational data. I’ve been interested in the area of causal inference in the past few years. In my opinion it’s more exciting and relevant to everyday life than more hyped data science areas like deep learning. However, I’ve found it hard to apply what I’ve learned about causal inference to my work.

Product Management for AI

Domino Data Lab

Skomoroch proposes that managing ML projects are challenging for organizations because shipping ML projects requires an experimental culture that fundamentally changes how many companies approach building and shipping software. Companies with successful ML projects are often companies that already have an experimental culture in place as well as analytics that enable them to learn from data. Pete Skomoroch presented “ Product Management for AI ” at Rev.

Data Ethics: Contesting Truth and Rearranging Power

Domino Data Lab

Also, data science work is experimental and probabilistic in nature. This Domino Data Science Field Note covers Chris Wiggins ‘s recent data ethics seminar at Berkeley.

Experiment design and modeling for long-term studies in ads

The Unofficial Google Data Science Blog

accounting for effects "orthogonal" to the randomization used in experimentation. For example in ads, experiments using cookies (users) as experimental units are not suited to capture the impact of a treatment on advertisers or publishers nor their reaction to it.

CEOs Must End Digital Business Planning Paralysis

Mark Raskino

Digital business is mostly about entrepreneurial risk and market experimentation – not just pre-determined design and waterfall project implementation. Something I’ve noticed in a few interactions this year, has been too much planning for digital business innovation.

6 Data And Analytics Trends To Prepare For In 2020

Smart Data Collective

Quantitative analysis, experimental analysis, data scaling, automation tools and, of course, general machine learning are all skills that modern data analysts should seek to hone.

In 2018, Data Will Put the Human Back into Human Experience – Part 2

Kirk Borne

Two additional scientific approaches that we recommend are: dataify your mobile strategy, and develop a culture of experimentation. Experimentation is key to getting it right.

The Secret to Jumpstarting Your AI Strategy

Sirius Computer Solutions

The kit helps facilitate clients’ AI adoption journey from experimentation to production. Organizations across all industries are seeing the value and competitive advantage of having an artificial intelligence (AI) strategy.

MNIST Expanded: 50,000 New Samples Added

Domino Data Lab

2018 , 2019 ], the rediscovery of the 50,000 lost MNIST test digits provides an opportunity to quantify the degradation of the official MNIST test set over a quarter-century of experimental research.” This post provided a distilled overview regarding the rediscovery of 50,000 samples within the MNIST dataset. . MNIST: The Potential Danger of Overfitting.

Data Mastering at Scale


Novartis is mastering experimental data from wet Chemistry and Biology experiments. Data mastering (sometimes called Master Data Management or MDM for short) is now 15 years old. It arose because enterprises have been creating independent business units (IBUs) for a long time with substantial freedom of action. This allows IBUs to be as agile as possible; otherwise, every decision must go through headquarters decision making.

Predictive Analytics World 2019 – What I Learned and What I Said

Decision Management Solutions

Her presentation really showed the importance of persistence, experimentation and lateral thinking in developing an analytic solution. I presented on Backwards Engineering – planning Machine Learning (ML) deployment in reverse.