In his latest book, “Thank You For Being Late,” Thomas Friedman highlighted 2007 as one of the most pivotal years in technology: 2007 saw the birth of Hadoop, the iPhone and Amazon’s Kindle for instance. My bet is that 2018 will start a new era for the technology space, very much like 2007 did. I see two reasons for this: the cloud will change the game because it has become a measurable strategy for Microsoft, Google and Amazon. Big Data will be less about technology. It will be more about management practices and processes. And data engineers will be forced to rethink their world as the enterprise mindset for analytics excellence shifts.
However, there is ‘a catch’ embedded in each of these predictions. Let me try to convince you of my beliefs and let me know if I don’t.
The Cloud Fallacy
You’ve heard it before: the Cloud is huge. Microsoft, Amazon and Google have built multi-billion dollar businesses because enterprise CIOs are betting big on it. Just a few years ago, enterprises confessed they were ‘dabbling’ in it. This year, they are ‘all in”. I recently invited to attend the banking technology conference where executives of a large multinational bank spelled out the change when they told me: “The Cloud and nothing else”.
I wrote last year about how the Cloud model was disrupting the economics and the deployment requirements of on-premises vendors. I explained that many data management vendors would struggle to adapt to the Cloud and support their prospects’ evolving needs. I called it the “Cloud fallacy”.
This year, I want to bring your attention to the risk of the “Cloud Lock-In”. We’ve seen this movie before: in the old enterprise data management days, vendors that had started out selling database products, progressively moved up the stack, acquired or built applications to keep people engaged with their data warehouse and ultimately ‘locked’ customers into their suites. While CIOs liked the fact that they had one vendor to go to in case of issues (the “one throat to choke” theory), they also suffered from the fact they had one vendor to go for licensing. What was convenient at first, created less flexibility and leverage for CIOs.
If you missed this “lock-in” trend, look at Oracle’s acquisitions over the past decade and you’ll see what I mean…
Why this matters: as you start embarking on the “all cloud” journey, beware of the “lock-in”. Not every cloud is created equal. Amazon’s cloud has different options than that of Microsoft or Google. Each integrates differently with the rest of your environment. Most integrate best with their own stack. You will want to architect your technical environment in a flexible manner. Learn from the last 2 decades: you might likely want to switch from one vendor to the next over time so you’ll want to make sure that the work you’re doing for one cloud is portable across all other applications. One of our customers moved from a traditional data warehouse to a large on-premises Hadoop cluster, then to the Google Cloud in less than 9 months. That’s less time than it used to take to build a data warehouse in 2008. So, in 2018 and onwards, educate yourself around the concept of semantic layers and don’t get locked in!
Big Data, Big Schmdata
In 2008, Hip-hop band The Black Eyed Peas came up with catchy tunes that urged their fans to get ahead of their times. One of the lyrics that stuck with me was Fergy’s claim: “don’t be 2000 and late”.
When Big Data became big in 2008, enterprises started to hire data scientists and data engineers. In 2009, Hal Varian, chief economist at Google, claimed the sexy job in the next 10 years would be a statistician. Enterprises, for fear of being “2000 and late” on-boarded scientists galore and instructed to “start coding”. Some the applications that came out of that work were cool. But many CIOs were left with poor results: hiring tens, hundreds or sometimes even thousands of scientists to support the ever-accelerating data needs of enterprise employees, simply didn’t scale.
Industry analysts such as Gartner or Forrester will tell you that machine learning and artificial intelligence will come to save the day. That’s all well and good. But, when I meet with some of my most customers’ successful chief data officers, the first thing they talk to me, is NOT technology. It’s organization structure and the mindset change required to win. Forward-thinking CDOs talk about their journey to “Insights-as-a-service” and how they are building the “Amazon of Big Data Analytics” inside their corporate walls.
Why this matters: I learned about the concept of the “Amazon of Big Data Analytics” from Joseph DeSantos, VP Data Analytics at TDBank, and one of our customers. The way he articulated his vision for the center of analytics excellence was quite a contrarian. He explained that, in the past, the enterprise I.T. group’s raison de vivre was to “full-serve” the business. This meant that I.T. hired staff to lay out data infrastructure, build datamarts, and deliver fully baked reports and dashboards to business people, based on their requirements. The “self-service” data analytics area saw business users take control of the construction of reports, even sometimes the building of datamarts, data extraction and transformation tasks that were traditionally owned by I.T. This threw the enterprise for a loop. While CIOs liked to enable the business, they realized quickly that enabling the business with tasks they weren’t fully trained to do and data they weren’t technically entitled to see, meant all hell broke loose. If you think about I.T. as the owner of a restaurant, “self-service” was the equivalent of inviting guests into the kitchen and letting them use professional-grade appliances.
What CDOs have now realized is that they need a better way to enable the business and protect the enterprise. They want to hire people who think of themselves as ‘educators’ of the business. They want to build systems, processes and organizational structures that let employees gain agility and freedom, while the employer firm keeps control over their governance framework. In “restaurant speak” this would be the equivalent of a “salad bar” I suppose.
So, while you’ll hear a lot about machine learning, artificial intelligence and other cool trends, I suggest you keep an eye on IaaS (Insights-as-a-Service). According to Forrester, that market will double, with 80% of firms relying on insights service providers for some portion of insights capabilities in 2018. All technologies are welcome but they need to be deployed as part of the right people-structure and they need to be applied for the right use case.
