Check before you spread news of the world… On Buzzfeed’s analysis of Budget news

Over on BuzzFeed, Mark Di Stefano has had a crack at an analysis of the public’s engagement with media stories about the Budget.  It’s an interesting read.  I’m not really sure what the point of the article is, but there are a few aspects to it which are worth interrogating in a bit more detail.

The analysis starts by taking a sample of the top 100 stories ‘shared’ (not ‘read’) on social media about the Budget (or Federal politics… it’s not clear) over the past three weeks.   In big massive, BuzzFeed font, it lets us know that the top story (with 33,800 shares — not reads) is a story about Jacqui Lambie suggesting that Federal politicians should get drug tests.

First up, it is difficult to know how to interpret this.  Is 33,800 shares a lot?  The text says that it is ‘astronomical’, but it is not clear why.  What is the average number of shares for a story?  Are political stories shared as much as non-political stories?  More?  And why are we looking at shares instead of reads?

What makes the figure even more difficult to interpret is that the 3rd, 4th, and 5th most shared articles are all about the same story: Senator Waters breastfeeding her infant in Parliament.  So which was the more shared story?  And then we’re back to the original problem: what does it mean for a story to be shared?  I don’t mean this in the technical sense (say, links on Twitter and Facebook) but in an interpretive sense.  If something gets shared a lot on social media, does that mean people are engaging with the story (and isn’t a read count a better indicator of that?) or is there something quirky about the social media environment in which a (frankly) dopey story about a dopey politician’s dopey opinion about drug testing can thrive?

And what does it mean that the ABC version of the story was shared significantly more than other versions of the story?  It’s 33,800 shares eclipses Fairfax’ 11,200.

The profile of the count seems to be overlooked in the analysis.  There is a freaking massive long tail to this data.  The fifth most shared story is less than half of the most shared story.  It doesn’t halve again within the top twenty.  Indeed, the 20th most shared story is 10,200 which is less than the population of my rural home town.  To put this even more into context: that’s about a quarter of the student population of the University of Melbourne.

So we are dealing with tiny, tiny numbers.  The difference between the 20th most shared story and the 20th is a paltry 1,700 shares.

It therefore defies sense that the analysis turns to the proportion of stories within the top 100.  BuzzFeed tells us that 15% of stories in the top 100 were about the drug testing policy.  15% of 100 is 15, so it’s not entirely clear what is being argued with the percentages.  Further, only two of those stories appears in the top 20, so the remaining 13 had share counts of fewer than 10,200.

So who cares?  What is the data actually saying?  And why are we using such freaking hideous colours in a pie chart to represent the data?

BuzzFeed then claims that Australians LOVE satirical news sites stating that, if they had been included, 8% of the 100 most shared stories would have been from the hilarity merchants in Australia’s comedy scene.  8% of 100 is eight.  So eight stories would have made the top 100.  And one of those stories only clocked in 4,300 shares.  More people in Australia were hospitalised for falling off a ladder than shared that story.

But the most baffling claim advanced is that there is no such thing as ‘Fake News’ in Australia.  It’s worth quoting the claim in full:

Not a single entry in the top 100 was from a source outside the “Mainstream Media”. No dodgy websites. No random blogs. Nothing.

The media panic around “fake news” does not bear out down under. Rather it’s been used by politicians to try and delegitimise news stories they don’t like.

There’s a lot to unpack here.  First, if there’s a ‘media panic’ about ‘fake news’, it is curious that BuzzFeed has been such a keen promoter of the panic.  Most people only know about ‘Dingo Twitter’ because BuzzFeed broadcast them to the rest of Australia.  Their news editor, Rob Stott, has been running a handful of stories about the right wing echo chambers on social media that are peddling fake news — some apparently (somehow) have even affected charities.  One imagines that Stott isn’t going to cool it with amplifying their message.  Perhaps those stories get shared a lot.

Second, it is difficult to know if it is the case that the selection of articles included looking for stories that were on dodgy websites, random blogs, and the like.  Given that 4,300 shares was enough to make the cut, it seems implausible that at least one crank website didn’t get close.

More importantly, there is nothing in the data to suggest that ‘fake news’ is ‘used by politicians to try and (sic) delegitimise news stories they don’t like’.  Nothing.

The belief that politicians are unfairly labelling things ‘fake news’ is a bit of an article of faith among many political journalists at the moment.  It’s part of the psychological defence mechanism which prevents a journalist from dealing with the criticism that they routinely get stories wrong.  They’re important for democracy.  They are speaking truth to power.  How could they possibly be peddling fake news?  Clearly, these lying bastards are lying to us.

The problem is not about the ontology of fake news, but the semiotics.  Why is ‘fake news’ such a useful phrase for politicians?  Because it resonates with the public who (following from BuzzFeed’s numbers) just aren’t engaging with political journalism.  There is widespread distrust of the media.  There is often outright hostility — such as when random members of the public take any opportunity they can to pull pranks on journos.  ‘Fake news’ exists as a concept because the public does not value the media.  When Fairfax went on strike, I sent a text to a person who does not engage with the media through social media to ask if they knew the strike was on: they did, but they didn’t think it would affect them because they got all their news from the television.  This is the reality of Australia at the moment — people just don’t care about news outlets because they don’t see the value of them.

No number of shares will establish that fake news is not a problem.  Analyses which purport to show otherwise are misunderstanding the problem.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s