Juha Saarinen: Dangers lurk in crisis app

Friends and relatives of many of those in Paris during the bombings turned to social media to find out if their loved ones were okay. Photo / Getty Images

Facebook needs to look hard at Safety Check before using it in disasters

Last Saturday started off great with perfect late spring weather in New Zealand, and then Paris happened.

Like many others, I first heard about the terrorist attacks on social media. Tweets about the horrors taking place scrolled past, with people linking to news reports about the shootings.

On Facebook, an acquaintance in Paris living in one of the areas where the murders took place wrote about her shock and fear at the events, but it was a relief to see that she was safe.

A very good friend of mine was informed via Facebook that her friends in Paris were safe. Not just once, but through five separate notifications, and not from the friends themselves but through Facebook's data mining algorithms.

This is Facebook's Safety Check in action, a feature that's normally only used for natural disasters. Facebook founder Mark Zuckerberg said it had been turned on for Paris, and later, for Beirut too, which had also suffered many casualties in a wave of attacks, like those in France, said to be by Islamic terrorists.

Although as Facebook vice-president of growth (yes, his actual title) Alex Schultz is correct in saying, communication is critical in times of crisis (tinyurl.com/nzh-schultz), I'm in two minds about Safety Check.

On the one hand, yes, for those worried about family, friends and relations in a crisis area, it's great to hear that they're unharmed. Safety Check is a relatively simple app that asks if you're okay - or not - and notifies your friends of the answer, which in a crisis I'm sure anyone would want to have.

On the other, it's Facebook, a giant, global commercial entity that sells user-generated data to advertisers, and which has stumbled over several privacy bumps along the way. Should the social network, by dint of its enormous number of users, be in this position, instead of a different, not-for-profit trusted entity?

The system is an experiment, too: Schultz cautioned that Safety Check is a work in progress that was "hacked together" during the Japan tsunami and nuclear power plant disaster four years ago and had some fairly significant problems in its first iterations.

"We saw spam abuse with the tool in some launches, in others it has overwhelmed our notification systems because of the huge numbers of people wanting to tell their friends they are safe," Schultz said.

Schultz also freely admitted that no matter what Safety Check claims, it's impossible to know when someone is truly safe.

If Safety Check gives away your location on Facebook and elsewhere - and authorities have cautioned against sharing your location on social media - and it's captured, it could make matters worse for victims of terrorist attacks.

Could Safety Check be used to lead attackers to the friends of a victim?

What if an evil regime decides to block Facebook wholesale on its corner of the internet so that no Safety Check notifications make it through to friends and family who are worried sick?

Safety Check raises plenty of questions that Facebook hasn't done a good job of answering.

Facebook intends to deploy Safety Check more widely.

It could be on to something very good and useful from a humanitarian perspective, but Facebook needs to engage with others about it, and ask some hard questions about the app and itself, before doing so.