Covid-19 coronavirus: Pandemic masks thwarting face recognition tech

A study found that even the best commercial facial recognition systems have error rates as high as 50 per cent when trying to identify masked faces. Photo / NIST, AP

Having a tough time recognising your neighbours behind their pandemic masks? Computers are finding it more difficult, too.

A preliminary study published by a United States agency today found that even the best commercial facial recognition systems have error rates as high as 50 per cent when trying to identify masked faces.

The mask problem is why Apple earlier this year made it easier for iPhone owners to unlock their phones without Face ID. It could also be thwarting attempts by authorities to identify individual people at Black Lives Matter protests and other gatherings.

The National Institute of Standards and Technology says it is launching an investigation to better understand how facial recognition performs on covered faces. Its preliminary study examined only those algorithms created before the pandemic, but its next step is to look at how accuracy could improve as commercial providers adapt their technology to an era when so many people are wearing masks.

More reasons for wearing masks. Yes, you wear a mask to protect me and I wear one to protect you. And it protects the wearer too, potentially by reducing viral load. via @NYTimes https://t.co/yQ4CfBIRHY

— Leana Wen, M.D. (@DrLeanaWen) July 28, 2020

Some companies, including those that work with law enforcement, have tried to tailor their face-scanning algorithms to focus on people's eyes and eyebrows.

NIST, which is a part of the Commerce Department, is working with the US Customs and Border Protection and the Department of Homeland Security's science office to study the issue.

It tested the software by drawing digital masks onto the faces in a trove of border crossing photographs, and then compared those photos against another database of unmasked people seeking visas and other immigration benefits. The agency says it scanned 6.2 million images of about one million people using 89 algorithms supplied by tech firms and academic labs.

Under ideal conditions, NIST says the failure rate for the best facial recognition systems is only about 0.3 per cent, though research has found significant disparities across race, gender and age. Add masks and the failure rate rises to 5 per cent or worse. When confronted with masks, the agency says, "many otherwise competent algorithms failed between 20 per cent to 50 per cent of the time."

And look at all the creative options available: https://t.co/U2lXhofksz @BenjaminDooley@hudidi1

— Motoko Rich (@motokorich) July 28, 2020

Even before the coronavirus pandemic, some governments had sought technology to recognise people when they tried to conceal their faces.

Face masks had become a hallmark of protesters in Hong Kong, even at peaceful marches, to protect against tear gas and amid fears of retribution if they were publicly identified.

The Government banned face coverings at all public gatherings last year and warned of a potential six-month jail term for refusing a police officer's order to remove a mask.

Exclusive: According to a decade of data obtained by SBS under the Freedom of Information Act, asylum and refugee claims from Hong Kong residents have spiked in the past financial year amid ongoing unrest. https://t.co/jUUnu3QJiF

— SBS News (@SBSNews) July 28, 2020

Privacy activists, in turn, have looked for creative ways to camouflage themselves. In London, artists opposed to high-tech surveillance have painted their faces with geometric shapes in a way that's designed to scramble face detection systems.

Then came the coronavirus outbreak, when health experts around the world began strongly encouraging everyone to wear masks that cover the mouth and nose.

NIST's preliminary study says what masks people wear, and how they wear them, makes a difference to facial recognition systems. The results are mostly unsurprising: The more facial features that are covered, the harder it is to recognise the person beneath the mask.

- AP