With the push to automate increasingly complex data environments, along with a greater need to spur collaboration to deliver more effective and timely analytics capabilities, organizations are turning to DataOps as well as a series of related “Ops” methodologies. Will DataOps—eventually conjoined with DevOps, AIOps, and MLOps—help businesses compete in the digital era? Industry observers point out that these approaches are promising, but a lot of work lies ahead to achieve true collaborative and automated innovation in the data management space.
The XOps approach is not a practice adopted for its own sake, but to deliver enhanced capabilities to the business. “It is critical to understand that XOps is more of a cultural shift in thinking than a series of steps or actions that needs to be implemented,” said Danny Allan, chief technology officer at Veeam.
Importantly, these methodologies should help deliver advantages such as automation, standardization, cost control, and observability, said Eti Gwirtz, vice president of product at GigaSpaces. “Each encapsulates a wide range of practices and tools,” she cautioned. “What’s important to understand is the underlying motivation that drives usage of each of these methodologies. The main motivation for automation is quality and business continuity. Automated processes support repeatability of tested procedures and reduce the risk of having the business compromised.”
Observability is fundamental, Gwirtz emphasized. “Observability, whether of services or data, is key for obtaining a holistic view of operational workload performance. Specifically, data observability is key for business-related performance.”
There are productivity implications as well, Gwirtz added. “It’s easier to train one person to manage many workloads if they all follow the same standards. Otherwise, you may end up hiring a large number of experts, each managing their own separate workloads.”
LEADING XOPs
Regardless of the XOps strategy adopted, they all have something in common. “They all include consistent stages with automation incorporated into all steps of the pipeline,” said Allan. “This includes continuous development, continuous integration, continuous testing, continuous delivery, configuration management, operations management, and continuous monitoring. These methodologies lead to faster service delivery and improved communication across the different stakeholders in the data-based services. While DevOps tends to focus solely on the development and delivery of the services, DevSecOps expands on this model to include security in each of the phases of operations.”
Greater automation is a significant benefit, agreed Chris Bergh, CEO of Data-Kitchen. “Data professionals spend most of their time on manual processes to ingest, clean, and transform data in support of data operations,” he explained. “Automating these processes slashes maintenance costs and enables data scientists and engineers to focus on analytic insights that address business challenges. DataOps is the industry term for automating the processes related to data, and consolidating DataOps functions into a platform is called a DataOps process hub. Companies are increasingly conceptualizing their hub-and-spoke enterprise data enablement platform as a data hub. A process hub automates the processes and workflows that create the data hub. Whatever you call it, the quiet advent of the DataOps process hub represents the most underreported yet transformative trend in the data industry.”
Still, much of this is new to data operations as they stand today. Even DevOps—the granddaddy of XOps approaches—is still in its formative phases. “We are still at the beginning of DevOps being widely adopted in the industry,” said Frédéric Harper, director of developer relations at Mindee. “DevOps helps streamline the application lifecycle by automating different processes, providing better overall management—code, deployment, monitoring, incidents, and configurations—and creating bridges between departments. To ensure teams can move on to DevOps-specific methodologies, we need to make DevOps part of the workforce from Day One and shift away from making it an afterthought.”
DevSecOps is another approach data managers need to look at. Gaurav Rishi, vice president of product at Kasten by Veeam, calls the rise of XOps “a Cambrian moment” among cloud-native applications. “Over the next few years, these modern applications, developed as containerized microservices, will outnumber traditional applications that have been developed over the past 40 years,” said Rishi. “However, organizations need to operationalize these modern applications that are taking the portability advantages of Kubernetes. Applications are increasingly deployed in hybrid or multi-cloud architectures with very varied underlying hardware and software infrastructure—including databases. To meet the operational challenges of keeping these applications securely running, staying current with the latest updates, and dynamically scaling to meet global demand require enterprises to adopt DevSecOps practices and tools.”
RETURNS
What business results are now being seen from these XOps initiatives, and what are enterprises hoping to see? Industry observers report a number of benefits are already emerging. For example, MLOps provides organizations the opportunity to “streamline the process of getting machine learning models into production,” said Harish Doddi, CEO of Datatron. Previously, with traditional software and data management approaches, “companies might be able to get one or two machine learning models done a year, at great pain and with extreme inefficiency. But with MLOps, businesses can scale, and address multiple problems. They’ll be able to roll out improvements much faster.”
At an operational level, key metrics seen include a reduction in deployment latency and errors, said Bergh. “DataOps automation slashes deployment cycle time from weeks and months to hours and minutes. Through automated testing at each step of data operations, DataOps reduces the high rate of data errors in most data organizations to virtually zero. We are already seeing how data and analytics enable companies to assume a leadership position in their market space. In upcoming years, data agility will separate the leaders from the laggards.”
Properly implemented, XOps practices “lower human error, as we now automate multiple items, such as the deployment of applications,” said Harper. “They ensure repeatable processes—like running tests on different platforms or programming language versions—and give us the possibility to let automation do the boring but important tasks so that people can focus on where they bring real value. In addition, they facilitate processes such as rolling back database changes or application versions when something goes wrong in production.”