Fake news headlines fool American adults about 75% of the time, according to a large-scale new survey conducted by Ipsos Public Affairs for BuzzFeed News.
The survey also found that people who cite Facebook as a major source of news are more likely to view fake news headlines as accurate than those who rely less on the platform for news.
This survey is the first large-scale public opinion research study into the fake news phenomenon that has had a sweeping effect on global politics, and that recently caused a gunman to threaten a DC pizza place. The results paint a picture of news consumers with little ability to evaluate the headlines that often fly toward them without context on social media platforms. They also — surprisingly — suggest that consumers are likely to believe even false stories that don't fit their ideological bias. And the survey calls into question the notion — which Facebook has reportedly begun testing — that consumers themselves can do the work of distinguishing between real and fake news.
The new data comes from an online survey of 3,015 US adults conducted between Nov. 28 and Dec. 1. For more on the methodology, see the bottom of this article. A detailed summary of results to all questions can be found here. Additional calculations can be found here.
“The 2016 election may mark the point in modern political history when information and disinformation became a dominant electoral currency,” said Chris Jackson of Ipsos Public Affairs, which conducted the survey on behalf of BuzzFeed News. “Public opinion, as reflected in this survey, showed that ‘fake news’ was remembered by a significant portion of the electorate and those stories were seen as credible.”
The survey found that those who identify as Republican are more likely to view fake election news stories as very or somewhat accurate. Roughly 84% of the time, Republicans rated fake news headlines as accurate (among those they recognized), compared to a rate of 71% among Democrats. The survey also found that Trump voters are more likely to rate familiar fake news headlines as accurate than Clinton voters.
Top Fake News Headlines
In the survey, respondents were shown a random selection of six headlines — three true and three false — related to the election. Those six were drawn from a list of 11 headlines gathered largely from a BuzzFeed News analysis that compared the top-performing fake election news articles on Facebook to the the top-performing real election news articles on Facebook. Of the 11 headlines tested, five were false and six were true.
Respondents who said they recalled the story in question were then asked to rate the claim in the headline as "very accurate," "somewhat accurate," "not very accurate," or "not at all accurate."
Real news headlines received a higher overall accuracy rating than fake news. The respondents made 1,516 judgments about fake news headlines they’d recalled seeing or hearing about; 75% of the time, they thought those headlines were “somewhat” or “very” accurate. By comparison, they considered 83% of real news headlines to be accurate, based on 2,619 judgments.
Of the people surveyed, nearly 33% recalled seeing at least one of a selection of fake news headlines from the election. That compared to 57% of respondents who recalled seeing at least one of the real news headlines tested in the survey.
The fake news headline recalled by the largest number of respondents is the story from hoax website the Denver Guardian, “FBI Agent Suspected in Hillary Email Leaks Found Dead in Apparent Murder-Suicide.” Twenty-two percent of respondents people said they recalled seeing it.
The real news headline with the highest recall is a post-election CBS News story about Donald Trump saying he will not accept a presidential salary, “Donald Trump on Refusing Presidential Salary: ‘I'm Not Taking It.’” It was recalled by 57% of the 1,507 people shown the headline in the survey.
The fake news headline with the highest overall accuracy rating from respondents is “FBI Director Comey Just Put a Trump Sign on His Front Lawn.” Of the 186 people who recalled seeing it, 81% said it was very or somewhat accurate. (Go here to read a debunking of that claim.)
A false headline claiming a man was paid $3,500 to protest at a Trump rally also received a high accuracy rating, with 79% of the 348 respondents who recalled seeing it saying it was very or somewhat accurate.
One contributing factor to its spread is that the story was tweeted by Eric Trump, by former Trump campaign manager Corey Lewandowski, and even by Kellyanne Conway, the Trump campaign manager who led him to victory. (Conway later deleted her tweet.)
The two real headlines with the highest accuracy ratings from those who recalled seeing them were the the New York Times op-ed “I Ran the CIA. Now I’m Endorsing Hillary Clinton” with a 90% accuracy rating (among the 157 respondents who recognized the headline). A CBS News story about Donald Trump saying he will not accept a salary as president was also rated as very or somewhat accurate by 90% of the 860 respondents who recognized it.
Clinton Versus Trump Voters
People who say they voted for Hillary Clinton were less likely than Trump voters to view the claims made in these fake headlines as accurate, according to the survey. This may be partly due to the fact that the majority of top-performing fake news stories about the election on Facebook had a decidedly pro-Trump or anti-Clinton bent. However, it’s notable that a majority of Clinton voters still believed the fake news stories to be very or somewhat accurate.
On average, Clinton voters judged 58% of familiar fake news headlines as accurate, versus 86% for Trump voters. (These percentages are based on 434 judgments by Clinton voters and 634 judgments by Trump voters.)
A fake story about the pope endorsing Trump was seen as accurate by 46% of Clinton voters compared to 75% of Trump voters. The hoax about an FBI agent connected to a Clinton investigation being found dead was seen as accurate by 52% of Clinton voters and 85% of Trump voters.
Brendan Nyhan, a political science professor at Dartmouth college who conducts research into political misinformation, reviewed the data and said he is surprised by the high percentage of Democrats who rated the pro-Trump stories as very or somewhat accurate.
“It’s especially striking that both Democrats and Republicans think the stories are accurate in many cases,” said Nyhan. "Even partisan-motivated reasoning — which we might expect to make people question fake news that is harmful to their candidate — does not appear to protect people from believing in it."
Trump voters in particular gave a high accuracy rating to a story that falsely claimed he had sent his own plane to fly 200 US Marines home. That claim, which was debunked by the Washington Post, was given a boost in awareness when the website of Fox News host Sean Hannity reported it and Trump's campaign said it was true.
Facebook’s Role in Exposing People to Fake News
Though the survey does not prove a direct link between Facebook use and exposure to and belief of fake election news, it offers new data about the relationship between the platform and election misinformation.
People who said they rely on Facebook as a “major” source of news appeared to be disproportionately susceptible to fake news headlines. In the course of 553 judgments about fake news headlines they recognized, these respondents deemed the information to be somewhat or very accurate 83% of the time.
By comparison, fake news headlines were deemed accurate 76% of the time by people who consider Facebook to be a “minor” source of news (465 judgments), and 64% of the time by people who rarely or never use Facebook for news (498 judgments).
However, these percentages came from small groups of respondents and should be read cautiously.
“We have a lot more to learn about this topic, but it’s clear that Facebook in particular needs to take fake news much more seriously going forward,” said Nyhan.
The survey also reinforces how important Facebook has become as a source of news for Americans. A total of 23% of the more than 3,000 respondents list Facebook as a major source of news for them, with another 27% citing it as a minor source. Only CNN and Fox News had higher percentages of people who said they view those outlets as major or minor sources of news. (Both saw 27% of respondents list them as major sources of news.)
Of those surveyed, 47% say they visit Facebook multiple times per day, with another 15% saying they visit it once a day. YouTube was the second most popular social platform, with 20% saying they visit it multiple times per day, and 11% visiting it once per day.
“I don’t want Facebook deciding which legitimate political content appears in the News Feed, but I do hope the company can prevent 100% fake news from being such an attractive business opportunity to entrepreneurs and scam artists alike,” Nyhan said.
Notes and Methodology
Here’s the list of 11 election headlines tested in the survey:
Pope Francis Shocks World, Endorses Donald Trump for President, Releases Statement (Fake)
Donald Trump Sent His Own Plane to Transport 200 Stranded Marines (Fake)
FBI Agent Suspected in Hillary Email Leaks Found Dead in Apparent Murder-Suicide (Fake)
Donald Trump Protester Speaks Out: “I Was Paid $3,500 to Protest Trump’s Rally” (Fake)
FBI Director Comey Just Put a Trump Sign on His Front Lawn (Fake)
Melania Trump’s Girl-on-Girl Photos From Racy Shoot Revealed (True)
Barbara Bush: “I Don’t Know How Women Can Vote” for Trump (True)
Donald Trump Says He’d "Absolutely" Require Muslims to Register (True)
Trump: “I Will Protect Our LGBTQ Citizens” (True)
I Ran the CIA. Now I’m Endorsing Hillary Clinton (True)
Donald Trump on Refusing Presidential Salary: “I’m Not Taking It” (True)
Respondents were shown a random selection of six headlines, of which three were real and three were fake. If they said they recalled seeing or hearing about the headline, they were then asked to rate its accuracy as Very Accurate, Somewhat Accurate, Not Very Accurate, or Not At All Accurate. This was to ensure that the survey captured the overall awareness of real and fake headlines, and that it only tested perceptions of accuracy with people who said they were familiar with the headlines in question. As with any survey that relies on human memory, it’s important to note that some people may be mistaken as to whether they saw the headline or not.
Of the more than 3,000 people who completed the survey, 50% said they voted for Hillary Clinton, and 41% said they voted for Donald Trump. (The rest said either that they voted for another candidate or didn’t vote.) Thirty-nine percent said they are Democrats, 29% said they are Republicans, 28% said they were Independents, and 3% considered themselves to be “Other.”
For the more standard survey questions related to demographics and media consumption, we used questions developed by Ipsos and those previously used by BuzzFeed Research.
For survey results representing all respondents, the poll has a credibility interval of plus or minus 2 percentage points for all respondents. For more information about Ipsos Public Opinion’s online polling methodology, please go here.