The relationship between AI and big data is a two-way street, to be sure: Artificial intelligence success depends largely on high-quality data, and lots of it. Managing massive amounts of data and deriving value from it, meanwhile, increasingly depends upon technologies such as machine learning (ML) or natural language processing (NLP) to solve problems that would be too burdensome for humans to contend with on their own.
It’s a “virtuous cycle,” as Anexinet senior digital strategist Glenn Gruber told us recently. Whereas the “big” in big data once might have been seen more as a challenge than an opportunity, this is changing as organizations begin rolling out enterprise uses of machine learning and other AI disciplines.
“Today, we want as much [data] as we can get – not only to drive better insight into business problems we’re trying to solve, but because the more data we put through the machine learning models, the better they get,” Gruber explained.
[ Could AI solve that problem? Get real-world lessons learned from CIOs in the new HBR Analytic Services report, An Executive’s Guide to Real-World AI. ]
When big data meets AI: Use cases across industries
Let’s take a closer at one piece of that broader cycle: Examples of how AI can be used as a powerful lever with big data, whether that’s for analytics, improved customer experiences, new efficiencies, or other purposes. Consider these three significant possibilities for how AI and big data can become a productive pair.
1. Gleaning structured data from non-standardized sources
The challenges of big data can be plenty – storing it in a usable, cost-effective manner, for example. The “usable” piece can be particularly tricky when it comes to unstructured data, which accounts for the lion’s share – 70 percent or more, according to some estimates – of enterprise data. (When people talk about how big data will inevitably continue to get bigger, unstructured data is the big driver of that growth.)
Turning unstructured information into usable formats can be a beast of a chore for humans, especially in repetitive (but entirely necessary) back-office operations.
Mathias Golombek, CTO at Exasol, points to invoice processing as a specific example that illuminates the broader possibilities of using AI to automatically extract structured data from unstructured (or non-standard) formats.
“An example [of how AI can be applied to big data] would be training a model that learns from scanned invoices and the historical data of extracted structured data: invoice ID, due date, recipient, etc,” Golombek says. “This information normally had to be interpreted by human beings since every invoice looks a bit different, has different names or languages. But if you use the historical data of thousands of invoices, it’s possible to create a model that could give you automatically the structured data by just scanning new invoices.”
This same principle of using AI to automatically extract structured data from unstructured sources could apply widely, not just to operational areas like finance or HR, but to the broad (and often unglamorous) category of enterprise content management. This is a potential boon for data analytics, robotic process automation (RPA) and other forms of automation, and other purposes.
“Organizations are using AI to change their most valuable asset – their content. Up to 90 percent of enterprise content is unstructured and growing [at a rate of] up to 65 percent per year,” says Anthony Macciola, chief innovation officer at ABBYY. “Most unstructured data goes unanalyzed, leading to valuable information being lost and unusable. With AI, organizations are transforming unstructured data into actionable information that can be used within intelligent automation systems. This enables business leaders to make better business decisions faster.”
2. Streamlining complex and bureaucratic processes
Where there’s big data, there’s often complexity and bureaucracy. Think sectors such as healthcare, insurance, and financial services, which have plenty of both. As a result, these industries are increasingly experimenting (if not moving full-speed ahead) with the potential ways AI technologies can be used to cut red tape and generally improve processes and outcomes amid complex requirements around compliance and other issues.
Let’s focus on the financial sector for a deeper example:
