The technology scans customers’ faces and compares images to those on the store’s databases of known offenders or suspects.
Māori AI and data ethicist Karaitiana Taiuru was not surprised to see Māori singled out in the trial and believed there were other Māori who had been affected but not come forward.
The systems Foodstuffs were using were trained on an international dataset of people and not on people in a New Zealand context, he said.
“We know from international data and international research that the systems are based on European-looking men, so it’s only in the last few years that the system has been modified to consider people of colour, women, men with beards etc.
“Anyone who is Māori, Pasifika, any person of colour is not going to be recognised by the system. That’s why it’s important that we have human double-checking [of] the system, which in this case failed.”
The woman was trespassed from the Rotorua supermarket despite offering three forms of photo identification to staff.
Taiuru said humans manually checking positive matches from facial recognition technologies often still relied on AI and not their own common sense.
The woman was shopping at New World in Rotorua. Photo / Andrew Warner
“It’s a phenomenon that’s identified in America where the staff rely on the AI more than their own personal judgment. So regardless of how obvious a mismatch is, humans still rely on the AI to be correct,” he said.
This bias could be reduced with appropriate staff training, which would have to be customised to the New Zealand population, he said.
Taiuru said as far as he was aware, there was no out-of-the-box training available in New Zealand yet.
If you were misidentified by facial recognition technology, it was best to complain to the organisation using it or the Privacy Commissioner, he said.
Figure NZ chief executive Ngapera Riley was from Rotorua herself and said it was outrageous to see Māori being profiled.
It was a clear example of why facial recognition technology might not be reliable at this stage, Riley said.
“I’m not surprised this happened in Rotorua with the supermarket trials, but these supermarkets, they’ve got a lot of money and access to resources so they really should be doing a better job at preventing these situations from happening,” she said.
It was a bad look when something like this happened in a high Māori population area like Rotorua, she said.
There were beneficial and concerning elements to biometrics and Riley understood why supermarkets and retailers were interested in the technology.
“The reason that supermarkets and retailers are implementing this technology is because they’ve got a huge increase in crime and their staff getting assaulted while doing their jobs... so they’re wondering can these new technologies help them with that.”
The office of the Privacy Commissioner is asking the public to have its say on a draft biometric code of practice.
New Zealand does not currently have special rules for biometrics.
Riley said biometric data included fingerprints, facial, voice, irises, palms and hand technology.
“It can also include things like keystroke patterns and the way someone walks.”
