Research
Publications
Overinference from Weak Signals and Underinference from Strong Signals (with Ned Augenblick and Eben Lazarus). Forthcoming, Quarterly Journal of Economics
Abstract: When people receive new information, sometimes they revise their beliefs too much, and sometimes too little. In this paper, we show that a key driver of whether people overinfer or underinfer is the strength of the information. Based on a model in which people know which direction to update in, but not exactly how much to update, we hypothesize that people will overinfer from weak signals and underinfer from strong signals. We then test this hypothesis across four different environments: abstract experiments, a naturalistic experiment, sports betting markets, and financial markets. In each environment, our consistent and robust finding is overinference from weak signals and underinference from strong signals. Our framework and findings can help harmonize apparently contradictory results from the experimental and empirical literatures.
JobFair: A Framework for Benchmarking Gender Hiring Bias in Large Language Models (with Ze Wang, Zekun Wu, Xin Guan, Adriano Koshiyama, Skylar Lu, Sachin Beepath, Ediz Ertekin Jr, and Maria Perez-Ortiz). Findings of the Association for Computational Linguistics: EMNLP 2024
Abstract: The use of Large Language Models (LLMs) in hiring has led to legislative actions to protect vulnerable demographic groups. This paper presents a novel framework for benchmarking hierarchical gender hiring bias in Large Language Models (LLMs) for resume scoring, revealing significant issues of reverse gender hiring bias and overdebiasing. Our contributions are fourfold: Firstly, we introduce a new construct grounded in labour economics, legal principles, and critiques of current bias benchmarks: hiring bias can be categorized into two types: Level bias (difference in the average outcomes between demographic counterfactual groups) and Spread bias (difference in the variance of outcomes between demographic counterfactual groups); Level bias can be further subdivided into statistical bias (i.e. changing with non-demographic content) and taste-based bias (i.e. consistent regardless of non-demographic content). Secondly, the framework includes rigorous statistical and computational hiring bias metrics, such as Rank After Scoring (RAS), Rank-based Impact Ratio, Permutation Test, and Fixed Effects Model. Thirdly, we analyze gender hiring biases in ten state-of-the-art LLMs. Seven out of ten LLMs show significant biases against males in at least one industry. An industry-effect regression reveals that the healthcare industry is the most biased against males. Moreover, we found that the bias performance remains invariant with resume content for eight out of ten LLMs. This indicates that the bias performance measured in this paper might apply to other resume datasets with different resume qualities. Fourthly, we provide a user-friendly demo and resume dataset to support the adoption and practical use of the framework, which can be generalized to other social traits and tasks.
The Fake News Effect: Experimentally Identifying Motivated Reasoning Using Trust in News. American Economic Journal: Microeconomics (May 2024, pp 1-38)
Abstract: Motivated reasoning posits that people distort how they process information in the direction of beliefs they find attractive. This paper creates a novel experimental design to identify motivated reasoning from Bayesian updating when people have preconceived beliefs. It analyzes how subjects assess the veracity of information sources that tell them the median of their belief distribution is too high or too low. Bayesians infer nothing about the source veracity, but motivated beliefs are evoked. Evidence supports politically motivated reasoning about immigration, income mobility, crime, racial discrimination, gender, climate change, and gun laws. Motivated reasoning helps explain belief biases, polarization, and overconfidence.
Media: Jean-Jacques Laffont Prize lecture (video); Times Radio UK (audio); MSN
Gender Differences in Motivated Reasoning. Journal of Economic Behavior and Organization (November 2021, pp 501-518)
Abstract: Men and women systematically differ in their beliefs about their performance relative to others; in particular, men tend to be more overconfident. This paper provides support for one explanation for gender differences in overconfidence, performance-motivated reasoning, in which people distort how they process new information in ways that make them believe they outperformed others. Using a large online experiment, I find that male subjects distort information processing in ways that favor their performance, while female subjects do not systematically distort information processing in either direction. These statistically-significant gender differences in performance-motivated reasoning mimic gender differences in overconfidence; beliefs of male subjects are systematically overconfident, while beliefs of female subjects are well-calibrated on average. The experiment also includes political questions, and finds that politically-motivated reasoning is similar for both men and women. These results suggest that, while men and women are both susceptible to motivated reasoning in general, men find it particularly attractive to believe that they outperformed others.
Polarization and Public Health: Partisan Differences in Social Distancing during the Coronavirus Pandemic (with Hunt Allcott, Levi Boxell, Jacob Conway, Matthew Gentzkow, and David Yang). Journal of Public Economics (November 2020, 104254)
Abstract: We study partisan differences in Americans' response to the COVID-19 pandemic. Political leaders and media outlets on the right and left have sent divergent messages about the severity of the crisis, which could impact the extent to which Republicans and Democrats engage in social distancing and other efforts to reduce disease transmission. We develop a simple model of a pandemic response with heterogeneous agents that clarifies the causes and consequences of heterogeneous responses. We use location data from a large sample of smartphones to show that areas with more Republicans engaged in less social distancing, controlling for other factors including public policies, population density, and local COVID cases and deaths. We then present new survey evidence of significant gaps at the individual level between Republicans and Democrats in self-reported social distancing, beliefs about personal COVID risk, and beliefs about the future severity of the pandemic.
Media: Wired, CNN (video), New York Times, Mother Jones, Economist, Reuters, Los Angeles Times, FiveThirtyEight, Newsweek, USA Today
Working Papers
Abstract: When people choose what messages to send to others, they often consider how others will interpret the messages. A sender may expect a receiver to engage in motivated reasoning, leading the receiver to trust good news more than bad news, relative to a Bayesian. This paper experimentally studies how motivated reasoning affects information transmission in political settings. Senders are randomly matched with receivers whose political party's stances happen to be aligned or misaligned with the truth, and either face incentives to be rated as truthful or face no incentives. Incentives to be rated as truthful cause senders to be less truthful; when incentivized, senders send false information to align messages with receivers' politically-motivated beliefs. The adverse effect of incentives is not appreciated by receivers, who rate senders in both conditions as being equally likely to be truthful. A complementary experiment further identifies senders' beliefs about receivers' motivated reasoning as the mechanism driving these results. Senders are additionally willing to pay to learn the politics of their receivers, and use this information to send more false messages.
Good News Is Not a Sufficient Condition for Motivated Reasoning. R&R, Review of Economics and Statistics (January 2024)
Abstract: People often receive good news that makes them feel better about the world around them, or bad news that makes them feel worse about it. This paper studies how the valence of news affects belief updating, absent functional and ego-relevant factors. Using experiments with over 1,500 participants and 5,600 observations, I test whether people engage in motivated reasoning to overly trust good news versus bad news on valence-relevant issues like cancer survival rates, others' happiness, and infant mortality. The estimate for motivated reasoning towards good news is a precisely-estimated null. Modest effects, of one-third the size of motivated reasoning in politics and performance, can be ruled out. Complementary survey evidence shows that most people expect good news to increase happiness, but to not systematically lead to motivated reasoning. These results suggest that belief-based utility is not sufficient in leading people to distort belief updating in order to favor those beliefs.
And Now for Something Completely Different
Tribone Tilings of Triangular Regions that Cover All but Three Holes. Discrete & Computational Geometry (February 2015, pp 466-477)