A video call with Facebook's Portal TV (the black camera below the TV). Photo / Supplied
A video call with Facebook's Portal TV (the black camera below the TV). Photo / Supplied
Facebook, which last month said it stopped using humans to review and transcribe users' voice messages, will resume that practice for some audio collected from its Portal video-calling device.
The move came as Facebook announced three models of its "Portal" smart camera for New Zealand, a year after the privacy-challenged social media company first released the product-line overseas.
The Portals will ship for NZ from November 5.
The Portal is a smart-display device designed primarily for video chatting, but not for checking Facebook - it is adding three more Portal products.
Facebook on Wednesday announced two Portal displays designed to look like picture frames, which will come in 8- and 10-inch sizes for $219 and $309, respectively. It's also making an entirely new kind of product called Portal TV, which is a camera-sporting black box that connects to a television, comes with its own remote and costs $259.
The picture frame version of the Portal. Photo / Supplied
The lineup also includes the $479 Portal+, which looks like a large tablet computer with a web cam attached to the top.
($NZD prices quoted. Facebook says Kiwis will get $80 off if they buy two or more Portals.)
When you're not using a Portal for Facebook Messenger or WhatsApp chats, it can act as a digital photo frame.
The Portal+. Photo / Supplied
And all portal's have Amazon's Alex voice assistant built-in, which can be used to control Amazon's Prime Video streaming service and answering queries about the weather and more a la Apple's Siri and Google's Assistant.
Facebook "paused human review of audio" around August. Bloomberg reported at the time the company hired contractors to transcribe private voice messages sent via its Messenger app. In that case, users had not been alerted to the possibility that their communications might be subject to human review. It was also unclear at the time that some of the clips Facebook had been collecting were coming from Portal.
Facebook confirmed Wednesday that it was indeed collecting audio from Portal users who make a request from the device using the command "Hey Portal." By default, those commands were recorded and stored on Facebook servers, and some of them were transcribed by contractors working with the company to improve the software algorithms used to understand the commands, according to Andrew Bosworth, Facebook's head of hardware.
That practice was paused last month at the same time Messenger stopped using humans to transcribe messages.
"We paused human review of the 'Hey Portal' voice interactions last month while we worked on a plan that gave people more transparency and control, including a way to turn it off," Bosworth said in a statement.
Portal is now reinstating human audio transcriptions but will offer consumers an option to turn off that service in a new version of its Portal software, which will be distributed to existing devices and its updated Portal lineup shipping in October.
The Messenger transcriptions are separate, Bosworth added, and that program is still on pause.
"The reason they're separate isn't because the back-end systems are separate, it's because the data is coming in from a different place," he told Bloomberg in an interview Tuesday. "And therefore you have a different kind of user expectation."
The controversial practice of transcribing user audio clips has gotten a lot of attention in recent months because of privacy concerns. Apple and Google have both suspended similar human transcription programs, and Bloomberg first reported in April that Amazon.com was transcribing some commands from its Alexa voice assistant without people's knowledge. Amazon now lets users opt out of that human review.
Facebook decided to reinstate this practice because it's important for training the company's software programs to accurately understand requests, Bosworth says. He's also aware that the idea of having humans review user audio is unsettling to many people.
"The consumer reaction the last several months to these practices, not just at Facebook but other companies, gave us insight into the fact that this was something people weren't entirely comfortable with or weren't sure about," he said when explaining the new privacy setting."
A spokesman for Facebook New Zealand said Portal offered simple and clear privacy settings.
You can disable the camera and microphone with a single tap or a sliding switch. A red light next to the lens indicates the camera and microphone are off and there's an integrated camera cover if you want to physically block the camera lens.
For added security, Smart Camera and Smart Sound use AI technology that runs locally on Portal, not on Facebook servers, he said.
If you have "Hey Portal" enabled, Portal listens for the phrase "Hey Portal." If it's detected, Portal sends a short audio recording and transcript of the "Hey Portal" voice interaction to Facebook. A trained team may review a sample to make our voice services smarter and more accurate for everyone. You can view, hear and delete any of your "Hey Portal" voice interactions in your Facebook Activity Log. You can also turn off voice storage in Settings anytime, which means that your voice interactions are not stored or reviewed.
Facebook will still collect and transcribe "Hey Portal" commands if users don't change the default settings. Portal's data usage policy states that the company does collect "voice queries and commands" after a user wakes the device with "Hey Portal." The policy does not say that those audio clips may be reviewed by third-party contractors.
The importance of audio transcriptions and recordings has increased alongside the rise of digital assistants like Amazon's Alexa and Google's Assistant. Tech companies improve the accuracy of their software by transcribing millions of clips, which help the machines learn language and speech patterns. The practice has, however, served up a new privacy trade-off: users want the help of smart assistants but not the threat that strangers might be listening to their private conversations or messages.
Facebook does not yet have an advanced standalone audio assistant to compete with the other tech giants, though its Portal device can carry out some basic commands after users wake it by saying "Hey Portal." For more complicated requests, Portal also comes equipped with Amazon's Alexa software.
Bosworth says that while Facebook is working to improve and further develop its "Hey Portal" software, it doesn't have any plans to completely replace Alexa on Portal devices with its own proprietary software, and Alexa is indeed present on Facebook's newly announced set of devices.
Facebook says WhatsApp calling via a Portal device is end-to-end encrypted.
