Artificial intelligence is 'shockingly' racist and sexist

MIT researchers looked at a range of systems, and found many exhibited bias. Photo / Getty Images

An MIT study has revealed the way artificial intelligence system collect data often makes them racist and sexist.

Researchers looked at a range of systems, and found many of them exhibited a shocking bias.

The team then developed system to help researchers make sure their systems are less biased, reports The Daily Mail.

"Computer scientists are often quick to say that the way to make these systems less biased is to simply design better algorithms," said lead author Irene Chen, a PhD student who wrote the paper with MIT professor David Sontag and postdoctoral associate Fredrik D. Johansson.

"But algorithms are only as good as the data they're using, and our research shows that you can often make a bigger difference with better data."

In one example, the team looked at an income-prediction system and found that it was twice as likely to misclassify female employees as low-income and male employees as high-income.

They found that if they had increased the dataset by a factor of 10, those mistakes would happen 40 percent less often.

In another dataset, the researchers found that a system's ability to predict intensive care unit (ICU) mortality was less accurate for Asian patients.

However, the researchers warned existing approaches for reducing discrimination would make the non-Asian predictions less accurate

Chen says that one of the biggest misconceptions is that more data is always better.

Instead, researchers should get more data from those under-represented groups

"We view this as a toolbox for helping machine learning engineers figure out what questions to ask of their data in order to diagnose why their systems may be making unfair predictions," says Sontag.

The team will present the paper in December at the annual conference on Neural Information Processing Systems (NIPS) in Montreal.

HOW DOES ARTIFICIAL INTELLIGENCE LEARN?

AI systems rely on artificial neural networks (ANNs), which try to simulate the way the brain works in order to learn.

ANNs can be trained to recognise patterns in information - including speech, text data, or visual images - and are the basis for a large number of the developments in AI over recent years.

Conventional AI uses input to 'teach' an algorithm about a particular subject by feeding it massive amounts of information.

Practical applications include Google's language translation services, Facebook's facial recognition software and Snapchat's image altering live filters.

The process of inputting this data can be extremely time consuming, and is limited to one type of knowledge.

A new breed of ANNs called Adversarial Neural Networks pits the wits of two AI bots against each other, which allows them to learn from each other.

This approach is designed to speed up the process of learning, as well as refining the output created by AI systems.