Business executives at mainstream corporations are quickly grasping a central premise that executives of data-driven companies have well understood -- the speed with which the enterprise can get value from its data matters. And, it matters a lot. Accelerating that timeframe, or improving on time to value, is or should be a core focus of any Chief Data Officer. As Mark Clare, former Chief Data Officer of HSBC’s and JP Morgan Chase’s retail divisions expressed it, “This is about speed and cost. Both are key requirements. Speed to market, and speed to value.”
Accelerating speed to market can be challenging enough, but when it is executed at the scale of data that corporate enterprises manage, the process can seem impossible. As Clare describes it, “I had oversight across numerous markets and hundreds of data sources.” In his efforts to fuel his firm’s insights, he was operating at a scale that was daunting. Clare continues, “At that time, both technology and legacy, manual processes wouldn’t scale to the challenge.” As a result, Clare had to forge his own approaches so that he could deliver timely results while he waited for the technology landscape to catch up. Solving the huge demand for analytics drawn from vast amounts of data sources while delivering insights in time to be actionable, is a core challenge for enterprises that seek to compete with agile, data-driven competitors. Chief Data Officers like Clare are turning to new approaches and technologies to tackle parts of the problem. These executives are learning that iterative, agile tactics yield the most dividends.
Next generation data engineering technologies and processes are part of the emerging Data Operations, or DataOps movement. The DataOps movement is about enabling organizations to deliver more comprehensive data to their business analysts and decision makers. This is done by applying some of the same techniques employed by the DevOps movement in software engineering to the disciplines of data modeling and data engineering. As Clare expains it, “DataOps really excites me because it creates formality around how to be a lean, agile data organization.” Interestingly, a recent survey noted that 73% of companies are planning to invest in DataOps initiatives to support their Artificial lntelligence (AI) and machine learning initiatives. The survey went on to indicate that a similar percentage of firms intended to hire professionals with DataOps skills as they seek to automate all aspects of data engineering.
Clare believes that DataOps represents the future. During his time at one large financial services firm, Clare undertook what was expected to be a multi-year project and delivered it in one quarter of that time by adopting an agile approach which streamlined processes through automation and agility. This enabled the business to deliver actionable insights across lines of business, and helped the enterprise optimize costs and increase revenue opportunities. The business benefits are clear. Today, DataOps practices are being used to accelerate the time that it takes to realize business value for a range of business activities. One example of how DataOps is having a business impact is in the transformation of fraud detection with consumer credit cards. Executives understand that fraud detection is an essential capability, but implementing a fraud detection system that doesn’t offend or inconvenience customers can be difficult. Denial of a credit card payment, or shutting down a credit card, represents a massive consumer inconvenience, with the result that customers may change banks in response. Banks that invest heavily in rules-based approaches to fraud detection may only see when certain rules are violated, without having a complete picture of the consumer’s pattern of spending. If a bank uses a set of rules to approve credit transactions, rather than employing dynamic analytics that are based on actual spending patterns, the result can be a bad outcome and an unhappy customer. As Andy Palmer, CEO of the DataOps firm Tamr puts it, “DataOps technologies look at the data itself to understand the data, rather than rely solely on declarative rules which may not be flexible enough to handle heterogeneity or ambiguity.”
Clare recalls an example drawn from personal experience. In 2010, while on a business trip, he learned that an Apple store had a received a limited inventory of new iPads. He purchased the device, but by the time he had reached the door he’d received a phone call and text message from his bank notifying him that his credit card had been shut off. While this transaction may not have made sense when viewed from the vantage point of being a large electronics purchase in a foreign city, when taken in the context of his total purchase and business travel history, it wasn’t out of line. DataOps presents an alternative. If the bank had engineered a system which dynamically incorporated new data to generate insights about Clare from his spending, rather than checking his transaction against static rules, it would not have needed to send a text alert, call his phone or lock his account, leading to a much better customer experience. By focusing on engineering repeatable, reliable and dynamic analytics, rather than static rules, enterprises are able to create a better picture. That engineering challenge is difficult, but embracing DataOps technologies and processes will allow firms to automate as many processes to create better analytics pipelines In this case, dynamic customer spend analytics reduce fraud detection friction with customers, in turn reducing churn.
DataOps approaches enable enterprises to derive value quickly and efficiently from their data assets. In the example of consumer spending, real-time resolution is the target outcome. As a result of more efficient and accurate data preparation and data integration, organizations can finally begin to rapidly address real consumer needs that accelerate the time in which it takes to deliver business value.
Randy Bean is an industry thought-leader and author, and CEO of NewVantage Partners, a strategic advisory and management consulting firm which he founded in 2001. He is a contributor to Forbes, Harvard Business Review, MIT Sloan Management Review, and The Wall Street Journal. You can contact him at rbean@newvantage.com and follow him at @RandyBeanNVP.
