Artificial Ignorance - To Err Is Human

    Matt Jones



    In March 2018, some argue, artificial intelligence killed someone for the first time. A self-driving car, which uses machine learning (a type of AI), took too long to conclude that the data from its sensors indicated a pedestrian. The result was a tragic loss of life.

    This is one of the most upsetting examples of “AI gone wrong,” but others also appear worrying. Another famous example is Amazon’s resume-assessing AI, which seemingly developed a sexist algorithm and started ruling out female applicants. AI is going rogue!

    Except it isn’t. In both cases, the AIs did exactly what they were trained to do. AIs do not know the value of human life or understand the concept of prejudice. They follow their training to make sense of the data they are given.

    The claims of failure suggest that the AI did something unintended, in the sense that a car’s brakes might fail, or an HR professional might mistakenly throw away a pile of resumes. But the AIs did not “go wrong” — they did everything right, according to what they were set up to do.

    That’s not to downplay the impact of AI missteps; some consequences have been disastrous. But pointing the finger at machines does not take us forward. We may indulge in a little schadenfreude when the world’s biggest companies mess up, if we like. But then, let’s move on and look at the reasons AI does not perform as hoped and see how we can do better.

    Raised By Humans

    AIs are built to interpret specific types of data and generate insights based on that data. If they are not built correctly, they will misunderstand the data. If they are fed wrong, incomplete or biased data, they will draw the wrong conclusions. Any AI output comes down to the humans who create and train them.

    Amazon’s AI was given past job applications and told to learn what factors translated into job offers. The AI ranked resumes by similarity to the data it was trained upon (i.e., past resumes and current employees). Being male was clearly a major factor, and the AI picked up on that.

    Uber’s self-driving AI learned that a certain set of parameters indicated a pedestrian. The set of parameters it was presented with — a woman wheeling a bike in the dark — was outside of its training, so it was unable to correctly classify what it saw in time. That’s not to say this AI doesn’t work; rather, it needs more training to reliably assess every possible scenario it might encounter.

    Building AIs That Actually Work

    The problem of AIs not producing the intended result is a real one, and the lessons of Uber and Amazon are valuable: If the AI is not designed and trained for the task, it will not work the way you want it to.

    There are two potential problems. The first is to identify and build an algorithm that is best suited to understand and interpret the type of data it will be using to generate the required answers. Although this is complex, organizations are improving. The customer-facing platforms of Uber, Amazon and Netflix are good examples of developing the right model to meet the task and data at hand.

    The second is gathering the right data and correctly training the AI, and this often poses the greater challenge. Real-world data is not as simple as often represented. The film ratings on which Netflix trains its algorithms are relatively tidy. Databases of resumes or traffic signals are far more complex and unpredictable, as well as being full of user biases, errors and missing parts.

    Training the best AI solutions demands huge amounts of high-quality, well-understood, clean data. Dealing with such data needs humans to be involved in the training, and specifically human experts who understand what the data represents. Both issues are often overlooked.

    Returning to the Amazon example, the AI did what it was trained to do: prioritized resumes that its training data showed led to job offers. Amazon does indeed recruit more men to technical roles, but this may be due to the number of applicants rather than gender preference. The AI doesn’t know this instinctively; it does what it is trained to do. It was a human design error, not that of a machine.

    So, how should Amazon have prevented this from happening? Their training regime and data sets should have been subjected to greater scrutiny by their HR people in collaboration with their AI experts. The hidden and implicit biases should have been caught and corrected for.

    The same human dimension is equally vital for industrial AIs. For example, a generic AI model may see a change of mechanical speed of a machine — say a drill — and conclude there is a problem. A drilling expert can say whether that change is due to an electrical fault or the substrate being drilled. With the right human training, the AI will recognize this, too, and make the right decision on whether to shut down operations, adjust settings or carry on as normal.

    Continuing oversight is also essential, constantly checking that AI results appear sensible to a human expert and modifying learned behavior to compensate for biases. Should the machine make a wrong move, this person is also the human backstop.

    AI can be hugely beneficial, but remember, it is just a tool. It must be designed, implemented and validated by people who truly understand it. The technology will work as expected, but only in the right hands. The way to stop tales of machines wrecking things is to have them trained by human experts who understand both the tools and what the data represents in the real world.

    By Matt Jones, lead analytics strategist at Tessella.

    Original source: Forbes Technical Council