Firms Must Overcome Human Barriers to Enable Data-Driven Transformation

How is data-driven business transformation driving better business outcomes, and what are the obstacles to that transformation? These are the core themes of NewVantage Partners 8^th annual executive survey, to be published on January 6, 2020.

Nearly 75 Fortune 1000 or industry leading firms are represented in this year’s survey, among them Allstate, Berkshire Hathaway, Capital One, CVS Health, Google, General Motors, Johnson & Johnson, and Met Life. C-executive decision-makers comprise 98.8% of the survey participants. 

Leading firms are pushing hard to realize business outcomes from their Big Data and AI investments. Yet, for most firms, challenges remain and progress against Big Data and AI returns must be viewed in the context of a multi-year journey with progressive stages of maturity.

While companies are certainly well on their way with their investments in Big Data and Artificial Intelligence (AI) capabilities, with 98.8% reporting active investment underway, what has been less understood is the extent to which companies are generating business results and measurable outcomes. Although 73.3% of the survey participants indicate that they are achieving measurable results, the precise nature of these results has been less understood. 

Data and AI investment are up, but the pace of investment is slowing

In recent years, it has become evident that investment in Big Data and AI initiatives is nearly universal among surveyed firms.  That trend continues unabated in 2020, as the percentage of firms investing in Big Data and AI continues to climb. The percentage of firms investing greater than $50MM is up to 64.8% in 2020 from just 39.7% in 2018. 

What is new is that the degree of urgency associated with last year’s investments in Big Data and AI has appeared to ease up considerably.

While 91.6% of executives reported that Big Data and AI investments were accelerating in 2019, nearly half – 46.9% — of those executives now report that these investments are being undertaken at a steadier pace. This may be a case of digesting what firms have already invested in, or it may signal a tapering of Big Data and AI investment for many organizations.

Firms struggle to become Data-driven as progress remains slow

Firms report measurable results from their Big Data and AI investments, with 70.3%% responding affirmatively.  Yet, the data underscores what is obvious to many – that even with progress in achieving measurable results, a vast majority of organizations – 73.4% — still experience business adoption of Big Data and AI initiatives as a challenge. 

Remarkably, even with record investment levels, organizations continue to struggle to become Data-driven. For example:

• Only 26.8% of firms have forged a data culture; 73.2% have yet to achieve this
• Only 37.8% of firms have created a data-driven organization; 62.2% have not
• Only 45.1% of firms are competing on data and analytics; 54.9% are not
• Only half of firms are managing data as a business asset; half are not.

Nor has there been notable improvement, as demonstrated by the flatline trend over the course of the past 4 years. So why is this? 

Organizations continue to struggle with people and process challenges

The principal challenges to becoming data-driven continue to be cultural – people and business process related — and not attributable to technology, which is prevalent and abundant.   Why are so many organizations struggling to successfully tackle these human factors? Perhaps this is an area that warrants greater management focus and remedy.

One solution that many organizations have attempted in efforts to address organizational alignment and responsibility for data initiatives is through the establishment and appointment of a Chief Data Officer or Chief Data & Analytics Officer.  Yet, for 72.1% of firms, the CDO/CDAO function remains an unsettled role — reflective of the dynamic and disruptive changes that Big Data and AI have triggered over the course of the past decade. 

Just 27.9% of firms report that the CDO/CDAO is successful and established, while 23.0% report that they are struggling with turnover, and the remaining 49.1% reporting that the role is nascent and continues to evolve.   

                                               ***

In our Foreword to this year’s survey, co-authored with Thomas H. Davenport, author of the landmark study Competing on Analytics, we observe that over the course of the 8 years during which this survey has been conducted, the results remain largely consistent, reflecting  “a field that is struggling to succeed despite massive investments in technology and applications”. 

We draw the conclusion that “continuing in the current vein or giving up on the objective of data-driven organizations and cultures isn’t really a viable option”, noting that this would result in “the eventual demise of legacy organizations in favor of digital native firms — a prospect that many executives have feared”.

However, there is a bright note. While we believe that human change represents a persisting challenge for most legacy companies, we are convinced that those firms that step up and change the paradigm of data management in a more human direction will emerge as the leading firms of the future.