How to select an AI pilot project: 5 criteria

To create a tangible, actionable starting point for AI in your organization, you need to identify a real use case you’ll test an AI solution against
770 readers like this.

We recently posed five questions to ask when identifying AI opportunities in your organization. Once you’ve found answers, what do you do next?

For most teams and companies, it makes sense to start with a pilot. Call it a pilot project or pilot initiative or pilot program – the name doesn’t matter so much as the purpose, which is to create a tangible, actionable starting point for AI in your organization. That’s necessary because most businesses are starting from scratch. Trying to do everything – or simply experimenting with no real goals attached – is likely to produce results that range from frustrating to non-existent.

“While it’s absolutely worthwhile to experiment with AI to see what’s possible, it’s also very difficult to do that without a real use case to test it on,” says Tom Wilde, CEO at Indico Data Solutions. “Organizations with large data science teams might have this luxury, but the large majority of organizations don’t.”

[ How does RPA fit in with AI? Read also: How to explain Robotic Process Automation (RPA) in plain English. ]

Enter the AI pilot: This is the real use case you’ll test an AI solution against. We asked Wilde and others experts for their insights on choosing the right place to begin. That led to five criteria for identifying a good AI pilot – if you can check these off, you’re on the right path.

1. You have a well-defined problem to solve.

If you’ve been frustrated by the progress of early AI experiments in your organization, it may be because those trials haven’t had a clear end-game.

“Pick a project or use case with an end result in mind,” Wilde advises. “Too many AI efforts begin as innovation or discovery projects without any real business outcome in mind. Almost all of them stall.”

So begin by identifying a business problem that AI might be able to help solve. Wilde points to automating a manual business process to speed up cycle time and optimize valuable resources as a general example. 

“It does not have to be big or strategic, but you need a defined problem to experiment on and learn from,” Wilde says.  

That word “outcome” is important here: It’s not enough to identify a problem or goal. You also need to define the desired results.

That word “outcome” is important here: It’s not enough to identify a problem or goal. You also need to define the desired results.

[ Read also: AI in 2019: 8 trends to watch. ]

“Once a problem is identified as being of high business value, it is important to clearly state the problem and the solution objective,” says Arti Garg, emerging market and technology director at Cray. “This will ensure that there are clear success metrics for the AI pilot, increasing the likelihood of broader organizational buy-in if the pilot succeeds.”

2. You have clear ways to measure the outcomes.

Let’s talk more about “success metrics” – you need to know how you’re going to define and measure them before you start, otherwise you’re taking the teeth out of even the most roaring success story.

“Pick an opportunity where you can define the desired state,” Wilde says. “Many organizations approach AI with the notion that it will tell them what the right answer is inside a large pool of data. In reality, AI is great at discovering what maps to or matches an already defined desired state.” 

Wilde shares an example: Let’s say your business problem is compliance-related. Your team spends countless human hours scouring contracts and other documents for possible compliance issues. That might be a good problem for AI (and sub-disciplines such as machine learning) to alleviate, but only if you teach your AI the parameters of compliance.

“If this is what compliant contract language looks like, AI can then automate the process of identifying which contracts are compliant and which are not,” Wilde says. “If you can’t define the desired state, don’t expect AI to do it for you.” 

“A successful pilot should have several phases of increasing gains towards the ultimate business goal.”

Amy Hodler, AI and graph analytics program manager at Neo4j, says the next step after identifying an inefficiency or other problem is to define the metric by which you’ll measure outcomes. By working with the business stakeholders directly affected to define what those are, you’ll lay the groundwork for a successful pilot that produces compounding benefits.

“For a pilot, it's important to be able to demonstrate measurable gains early and to have those agreed upon with key stakeholders,” Hodler says. “A successful pilot should have several phases of increasing gains towards the ultimate business goal.”

[ Want to learn more about AI's pressing questions? Read 10 TED Talks on AI and machine learning. ]

Kevin Casey writes about technology and business for a variety of publications. He won an Azbee Award, given by the American Society of Business Publication Editors, for his InformationWeek.com story, "Are You Too Old For IT?" He's a former community choice honoree in the Small Business Influencer Awards.