AI pilot projects: How to choose wisely

Early success with artificial intelligence can change skeptics to believers. Here’s how to select an AI pilot project that gets people on board
353 readers like this.

A few years ago, I was listening to a vendor pitch with a group of enterprise IT veterans. The sell focused on features of an intrusion detection software product and the value of artificial intelligence. The vendor said AI techniques allowed the product to automatically detect threats.

Discussing the pitch afterward, the general sentiment of the group was “I don’t buy it. There is no such thing as magic!” I agreed. As a student of enterprise software, I understood two key things:

  1. Vendors hype the “next great thing" to make it seem more valuable.
  2. Software is very specific. Code has specific instructions and performs exactly as told.

Based on our past software experience, the vendor pitch seemed like snake oil. In our world, software did not figure things out for you. It just followed the rules set for it.

[ Is RPA a kind of AI? See our related post, How to explain Robotic Process Automation (RPA) in plain English. ]

We did not understand the power of AI, which is turning traditional software development on its head. I’m now convinced that using AI will quickly become as common as today’s use of databases. Far from viewing it as snake oil, I now understand that AI is essential to bring businesses to the next level.

How to select the right AI project – and beat resistance

Rather than software performing exactly what it’s told to do, an AI system learns what it needs to do.

Selecting the right AI pilot project is a vital first step because success with AI is different than with traditional software development. Rather than software performing exactly what it’s told to do, an AI system learns what it needs to do. That is the opposite of what we are used to, and that creates resistance to investing in AI.

[ Is the problem a good fit for AI? Read also: How to select an AI pilot project: 5 criteria. ]

Start by finding examples of outcomes you want (and you need many of these examples) so the software can learn what it needs to do. This means you must be prepared to explore and be comfortable not knowing why or how something works.

The best way to curb your own resistance to AI is to work on a use case. For example, a recommendation engine project served as a starting point for one company’s AI experience. Initially, it was hard to get product managers to attend meetings and provide feedback. But as the project progressed and the managers understood the approach, they all identified multiple opportunities within their own area of responsibility and sought to launch their own AI efforts.

They began to recognize how machine learning could be useful in their roles. This is why selecting the initial AI project is vital. It demonstrates the power of AI and machine learning and how it can be applied broadly across the business.

AI is about changing routines, and that will always meet resistance. The recommendation engine successfully connected technology and people to show how AI can help people achieve objectives.

[ Want lessons learned from CIOs applying AI? Get the new HBR Analytic Services report, An Executive’s Guide to Real-World AI. ]

Think hard on business impact 

For the initial project, you must choose something with a sizable business impact. You also need a reasonable amount of accurate data for the project to succeed. And the stakes are high – it must succeed; if not, the chance of landing a second project is significantly less. 

For example, I worked with an insurer who decided to use AI to identify billing errors. The machine-learning system could absorb past human errors identified on invoices. Before this project, only a small number of invoices could be checked, using a randomly selected sample set.

At the start of this AI project, the ways in which it could help the business were not fully understood.

The system’s initial focus was simply to select a better sample, identifying invoices with a higher likelihood of error. Today, the company still has people reviewing invoices, but they are augmented by an intelligent system that selects the documents for them to review. And the system is continually learning and adjusting. Basic errors can now be identified automatically, and it can process 100 percent of the invoices – something that was not possible before.

At the start of this project, the ways in which it could help the business were not fully understood. This pilot demonstrated the power of AI, and that project has now spurred the team to keep going and explore other AI paths.

Don't expect perfection

Only an impactful project can create true believers. But do not make the mistake of expecting it to be perfect right away. It is like the evolution of self-driving cars – they won’t yet drive 100 percent of the time, but will take over when needed.

People will dismiss technology when they do not understand its impact. As in the example of me and my colleagues in the early days of AI, it is easy to be disillusioned by technology promises, but personal experience convinces even the most cynical. Early successes will help you overcome the pessimists.

[ How can automation free up more staff time for innovation? Get the free eBook: Managing IT with Automation. ] 

Mark Kirby is Capgemini’s North American Chief Technology and Innovation Officer as well as the North American Lead of Perform AI, Capgemini’s AI service portfolio.