The "Fake News" Effect: Experimental Design
The design gives subjects information such that a Bayesian would not infer anything, but motivated beliefs are evoked. Directional inference is attributed to motivated reasoning.
There are three steps:
Step 1: Elicit Subjects' Median Beliefs
The subject states a belief m about a factual question: "How did the murder rate change under Obama?" I elicit a median belief m, such that she believes P(answer > m) = P(answer < m) = 1/2.
Step 2: Give Subjects Personalized News
The subject receives one customized message from an unknown source: Either True News or Fake News. She doesn't know the source, and has to infer it from the message.
True News: Always tells the truth.
Fake News: Always lies.
The message either says:
G: "The answer is greater than [subject's median belief] m." or
L: "The answer is less than [subject's median belief] m."
Step 3: Elicit Subjects' News Veracity Beliefs
The subject is asked to predict P(True News | message). Because of the median elicitation, there's nothing to infer.
Prior (P(True)) is fixed.
Likelihood (1/2) is same for both G and L.
Median elicited, so subjective likelihood P(G | True) = P(G | Fake)
But motivated beliefs are different for G and L.
If subjects are more likely to trust a news source that reports G as compared to a source that reports L, this is evidence for motivated reasoning towards G.