The term "big data" first emerged fifteen years ago to put a name to the increasingly large, diverse, and complex volumes of data that could not be easily managed by traditional data management practices. In recent years, as digital transformation picked up steam, big data has emerged as a primary fuel for the journey.
“The ability to analyze vast amounts of structured and unstructured data to gain insights, often in real time, is what underpins most digital transformation efforts, as the insight derived through big data analytics is used to drive digitization and automation of workflows,” says Rahul Singh, managing director of IT and business services transformation advisory firm Pace Harmon.
[ Do you say no to innovation theater? Read also Digital transformation: 11 habits of successful teams. ]
In addition, digital transformation emerged in part as organizations sought to make the best use of these growing troves of data assets. “Digital transformation is about transforming your organization to base its decisions on data, and big data is the ability to capture all the available data an organization can produce or consume,” explains Todd Wright, head of data management at SAS. “Capturing all the available data – big data – is essential to digital transformation efforts.”
IT organizations can certainly leverage big data purely for reporting and process improvement purposes. However, Singh explains, “The true value comes from the ability to combine big data with digital transformation efforts to enable digitization and automation of entire operations to drive efficiencies and new business models.” As Prashant Kelker, partner for digital strategy and solutions at technology research and advisory firm ISG, likes to say: “Digital transformation is the path. Big data is one of the means.”
[ Get answers to key digital transformation questions and lessons from top CIOs: Download our digital transformation cheat sheet. ]
How big data reveals digital transformation opportunities
Big data, at its best, can shine a light on otherwise dark corners of the enterprise. “Large amounts of well-managed data will deliver a better understanding of operations, customers, and markets when integrated within an analytics or AI program,” says Wright of SAS. “The bottom line is that for digital transformation to be truly successful and achieve the best insights for business goals, as much data as possible is essential.”
Big data on its own is useless without a well-thought-out idea or program to make use of it. “Digital transformation provides that idea and program,” says Wright of SAS. “As for whether big data is required for digital transformation, the more data that goes into a digital transformation program, the better the results.”
When the two converge, real change becomes possible. As the number of IoT devices, wearables, smartphones, and other machine sensors grows, so too does the amount of data they generate – to an exponential degree. “The combination of this IoT data, big data analytics capabilities, and digital transformation allows companies to not just adjust in near real time to customer needs but also predict future behavior of their consumers,” says Singh of Pace Harmon.
Ronak Doshi, vice president at management consultancy and research firm Everest Group, notes that along with the growing proliferation of internet-connected devices, evolving data-driven digital business models, as well as an increase in globally connected business ecosystems and integrated value chains, demand that enterprises build a modular but cohesive digital platform, powered by big data gathered from a variety of sources.
“The business value (or return on investment) that an enterprise can generate from their investments in the digital capability platform will depend on their data value extraction capabilities,” Doshi says. “For large enterprises, the volume of this data is huge and is not managed through traditional business intelligence and data warehouse techniques.”
Aim for integration, not isolation
Digital technologies focused on deriving the most value from big data can enable IT leaders to build data hubs for aggregating and staging data from multiple sources, says Doshi. Many big data vendors offer pre-built analytics and machine-learning algorithms that IT leaders can leverage.
However, it’s critical that both big data and related digital transformation efforts are clearly defined for the particular organization and industry first. One manufacturing firm Kelker was working with, for example, had to first determine if its goal was to increase revenue with connected products or to cut costs with a more optimized and connected shop floor – or some combination of the two. Only then could IT leaders determine the best big data, IoT, and cloud strategy to achieve those ends.
“Digital transformation should be done with the goal in mind,” Kelker says. “Is it new revenues? Is it cost savings? Is it both? This charter for transformation helps to define the path and guide the implementation of technology.”
Too many big data initiatives still start within an IT or business intelligence organization and die out due to lack of business relevance, says Kelker. “Big data initiatives, executed in isolation, feel like solutions looking for a problem to solve,” he says. “Well-executed digital transformation gives technology (including big data initiatives) the glide path.”
Big data and DT are also helping enterprises better understand customer preferences and behavior in order to create more personalized and relevant experiences. Beyond the pure monetization possible with the introduction of insight-based products and services, Doshi says, companies are also combining big data and DT to design new products and services that boost the top line.
Successful organizations take a proactive approach
In some cases, big data efforts can actually hinder digital transformation, particularly if the data isn’t supported by a solid data governance program, according to Wright of SAS. “Organizations cannot simply just have access to more data and from more sources without metadata management, data quality, data catalogs, and assigned security and owners of the data.”
IT leaders who have the most success using big data in aid of digital transformation take a proactive approach, Wright says. They start with a strategy for data management. “For digital transformation to be successful, it must be based on trustworthy data.”
Those organizations that invest in data governance as well as advanced analytics and machine learning see the most benefits from the big data-DT combination – ranging from increased operational efficiency to improved customer experiences to increased revenues.
“Digital transformation of supply chains is a good example where the need to build operational resilience required enterprises embark on their digital transformation journey,” says Doshi of Everest Group. “The key element of this journey was to track the data about how materials , finished goods, and/or information assets are moving through the supply chain. Insights from data are helping enterprises enhance the efficiency of their entire value chain and improve functions like distribution, logistics, manufacturing, and sales.”
[ Culture change is the hardest part of digital transformation. Get the digital transformation eBook: Teaching an elephant to dance. ]