Can we trust AI to make decisions?

Artificial intelligence (AI) is going to make our lives easier by automating repetitive tasks, releasing resources to focus on higher-value tasks and removing the subjectivity from human judgement.

Yet, sometimes it goes wrong - people are denied visas, wrongly sent bills, or are stopped at airports because AI has mistaken them for someone else. Compounding this problem is the issue that should AI make a wrong call or deliver an incorrect result you cannot just ask it why. As Douglas Adams once said:

“The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at or repair.”

As private enterprises, solution providers and public organisations recognise the potential of AI. AI enabled platforms are now involved in many more decisions than ever before. How should they provide the necessary transparency and oversight that maintains trust?

The importance of understanding AI decision-making

An essential first step is to understand how AI came to a decision. This is not an easy task, you need to go back and examine the potentially massive volumes of complex training data and determine unwanted biases in the algorithm or hidden layers in the neural network.

This issue of transparency is compounded further when the AI is embedded within a platform that the organisation has purchased from a third party. Want to add another layer of complexity? The AI enabled platform may be hosted in the cloud or leased from a platform service provider, further limiting access other than that intended by the suppliers.Make Your Digital Transformation Transformational – Download Our Guide to  Maximising AI’s Value

The 3 vital steps to building trust with AI:

Step 1: Design AI to meet the need directly

Until artificial general intelligence is a reality, AI needs to be customised directly to the task at hand. The business or organisational context is often not given enough significance, particularly in off-the-shelf, AI enabled platforms.

AI should be designed not just by technical teams, but by people who understand the underlying data and what it actually represents within this business context. We see far too many data projects focused on clever algorithms and advanced technology without consideration of the actual need of the people who need to consume the insights generated.

An AI enabled data discovery platform will see a pattern between a behaviour and a common outcome (whether that be between spending patterns and credit card fraud, or overheating and impending engine failures). Someone who understands the context will build a significantly more sophisticated and accurate algorithm to understand when that behaviour is normal and when it requires intervention. In short the areas and situations where oversight is required are understood in the design phase and can be built into the business processes that use AI.

Step 2: Apply rigorous focus on training data 

Machine learning, deep learning and neural networks – the toolset of AI – are only as good as the data that they are trained upon. Training data can often hold unseen biases – as shown by the embarrassing beauty contest AI which was trained to recognise beauty using images of white people so rated white people as more attractive. There is a need for a scientific approach which scrutinises the training data sets, accounts for AI explainability, and designs experiments to eliminate these otherwise hidden biases.

Step 3: Get the right expertise 

We now know that AI does not end with the algorithm (see step 2) – there needs to be an ongoing oversight from people who understand what the AI generated insights from data are telling them. It is not reasonable to expect a government official to understand complex deep learning topologies and subsequently dive into them when someone raises a concern. But a data scientist who knows the data, algorithm and the context can quickly identify if an error was made and take corrective action, an immediate intervention whilst also using that feedback to update the efficacy of the model.

AI Regulation

Recent calls for a watchdog to regulate AI development to be set up should be welcomed, in theory. But the needs for oversight of an AI enabled decision informing machine is not the same as the needs to oversee a human decision and needs a different level of informed and considered approach. 

Maximising the value of AI

Incorrect billing and unpleasant airport experiences are bad enough, but AI is helping to design and build the planes, manufacturing plants, nuclear reactors, and assess the safety of new medicines. 

AI is now being used in higher impact roles, with higher stakes and providing answers to more complex and important questions than ever before. And as the barrier of entry to AI is lowered, this list is growing, rapidly. Put simply, AI has the power to redefine business capabilities, enabling innovation and greater cost efficiency. With this in mind, how can organisations maximise the value of AI as the technology grows increasingly innovative? Read our AI guide, here

Pull Away from the Pack with Our Guide to Maximising AI’s Value


Subscribe to our newsletter

Join over 5,000 thought leaders and experts in R&D, science, engineering and business, giving you access to opinions, case studies and the very latest thought leadership.