What did people in the Facebook comments sections of major news websites have to say about the last month of Covid announcements? A new piece of social media sentiment analysis seeks to answer that very question.
The comments section on Facebook is often seen as a toxic place, but a new piece of sentiment analysis shows that is not always the case. And it suggests an unexpected bit of positivity around the country's Covid-19 response.
Anusha Rai, a data analyst at Auckland digital marketing agency Double, has been trawling through the Facebook comments on major news outlets' live streams of major Covid-19 announcements.
To gather up the sentiment, she used an open-source software tool called Vader, which is basically used to extract opinions from written text. It uses machine learning and natural language processing to do that, but requires some human input to make it work properly.
Rai used the tool to sort comments into positive, negative and neutral groups, and then compiled an average out of that. From the start of the re-emergence of community transmission until the end of August, that average sentiment generally came out positive.
"The data shows a clear bell curve shape, indicating that sentiment grew to a peak on 21-23 August before again rapidly dropping. The peak was the time the Cabinet was meeting to make a decision on the extension," said Rai.
Doing this sort of analysis isn't as simple as just plugging thousands of comments into a spreadsheet. After all, social media comments aren't written like a thesis or novel.
Can the tool handle sarcasm, or local idioms like "mean" meaning good? No, said Rai. "I actually had to manually go through a lot of the comments – it does not do well with sarcasm."
"That's been a pretty big problem with a lot of natural language processing, because sarcasm is quite hard to detect, so that's definitely one of the drawbacks. But Vader is also specific for social media, so it's pretty good at picking up social media sentiment in particular."
Among other features, it can also pick up emojis, and context. "In the context of the sentence, it's not just like the word 'mean' being bad, it does pick up on the whole context of the sentence."
Interestingly, Rai says a lot of negativity came in response not so much to the content of the announcement itself, but in response to other users. "Looking at the comments, a lot of the more negative comments were spam, or trolling and trying to get a rise out of people. The majority of them were quite neutral, just stating a point, or what was happening."
Sometimes a negative comment would then provoke dozens of other negative comments addressed to the original comments in response.
Rai's research looked at a few other streams in detail, relating to specific events. She also addressed announcements around the election move, mandatory masking, going back into level three lockdown in Auckland, and then coming out of it.
As political events, Rai also analysed and averaged out the sentiment that was directed towards both Labour and National as parties, and towards Jacinda Ardern and Judith Collins as leader. "Using the comments from the 1pm updates I extracted the compound score for every sentence that mentioned Labour, National, Judith, Jacinda and then took an average for each of them for each of the days," said Rai.
In general terms, the sentiment still came out largely positive for all.
That data was also averaged out as a head to head on sentiment. While both leaders generally saw overall positive responses, on most days Labour and Ardern had more favourable sentiment than National and Collins. The one exception was August 17, the day on which the election was delayed.
So what is the fundamental usefulness of this analysis? Rai says part of it was about figuring out how New Zealanders felt about the return to Covid-19 restrictions, particularly after seeing a lot of negativity towards the country in the social threads on international news platforms.
"I was reading the comments on international news articles, and it was quite interesting – there were a lot of Americans being quite negative towards New Zealand, but often making stuff up, which I found quite interesting. I wanted to see what the New Zealand comments were like, and seeing if maybe there was a difference in sentiment over this lockdown, as different things happened."
And for Rai, there was also the motivation of trying to gauge what regular people were thinking, at a time when conversations about Covid-19 in the media were being dominated by a range of loud voices with large platforms. It's one of the big strengths of social media comment analysis, because it captures the views of those watching along to major events at home.
