Steven Levitt is an American economist, and the coauthor of Freakon...
Expected utility is one of the first theories of decision making an...
Prospect theory is a theory of behavioral economics/finance develop...
Here is the website, which is still up: https://www.freakonomicsexp...
This is an important, surprising discovery: "Those who were instruc...
Status quo bias: an emotional bias; a preference for the current st...
Really interesting finding: "there appears to be a causal impact of...
This is key to the study design and the validity of the results: "I...
"The coin-flipper’s ex ante assessment of how likely he or she is to...
"In contrast, under the assumption that the only channel through wh...
This is indeed a large potential bias... this study will need to be...
"Summarizing the discussion above, it is likely that the first-stage...
Review of Economic Studies (2020) 0, 1–28 doi:10.1093/restud/rdaa016
© The Author(s) 2020. Published by Oxford University Press on behalf of The Review of Economic Studies Limited.
Heads or Tails: The Impact of a
Coin Toss on Major Life
Decisions and Subsequent
Happiness
STEVEN D. LEVITT
University of Chicago and NBER
First version received November 2017; Editorial decision October 2019; Accepted April 2020 (Eds.)
Little is known about whether people make good choices when facing important decisions. This
article reports on a large-scale randomized field experiment in which research subjects having difficulty
making a decision flipped a coin to help determine their choice. For important decisions (e.g. quitting a
job or ending a relationship), individuals who are told by the coin toss to make a change are more likely
to make a change, more satisfied with their decisions, and happier six months later than those whose coin
toss instructed maintaining the status quo. This finding suggests that people may be excessively cautious
when facing life-changing choices.
Key words: Quitting, Happiness, Decision biases
JEL Codes: D12, D8
1. INTRODUCTION
In every life, there arise difficult decisions with potentially far-reaching consequences on lifetime
utility: whether to quit a job, seek more education, end a relationship, quit smoking, start a
diet, etc. Expected utility maximization is the workhorse economic model for thinking about
such choices. Behavioural economics offers a host of alternative descriptive models of decision-
making, e.g. prospect theory, hyperbolic discounting, and the sunk cost fallacy. Yet, from an
empirical perspective, economics has almost nothing to say about whether or not people are
actually making good choices when it comes to their most important decisions.
1
1. There is, of course, a rich experimental literature exploring individual decision making under uncertainty. For
surveysof this enormous literature, see Camerer (1995), Smith (1994), and Chaudhuri (2011).A notablerecent contribution
to decision-making under uncertainty is Gneezy et al. (2015). Most of this literature focuses on low-stakes decisions.
Slonim and Roth (1998) and Andersen et al. (2011) explore decision-making in a high-stakes dictator game. In recent
years, field experiments exploring decision making in natural environments have become more common (Bowles et al.,
2001; Gneezy and List, 2006; Dellavigna, 2009; Levitt and List, 2009), but most of these have investigated relatively
The editor in charge of this paper was Nicola Gennaioli.
1
Downloaded from https://academic.oup.com/restud/advance-article-abstract/doi/10.1093/restud/rdaa016/5834495 by guest on 31 May 2020
2 REVIEW OF ECONOMIC STUDIES
One reason that so little is known about these important decisions is that researchers do not
generally have the power to randomize people into treatments that compel them to, say, quit
their jobs or leave their spouses. Even if it were possible to choose 1,000 married couples from
the general population and randomly force 500 of those couples to divorce, it would not be
particularly informative. Such a study would tell us about the average treatment effect of divorce.
What we really care about, however, is the impact on the marginal decision maker. It would not
be surprising if getting a divorce would have a devastating impact on the infra-marginal married
person. A much more interesting question is whether divorce, ex post, will be the right choice for
someone teetering on the edge of ending a relationship.
2
Even if one found such a group of individuals who are close to indifferent between remaining
married and getting divorced, an ex post comparison of the happiness of those who do and do not
make a change still would not have an easy causal interpretation, because the people who make
a change will systematically differ from those who do not on many dimensions. To convincingly
answer the question, a researcher would not only need to find large numbers of these marginal
individuals, but also, through some sort of randomization, influence their important life choices.
That is what I do in this study. I created a website called FreakonomicsExperiments.com.
On the website, individuals who are having a difficult time making a life decision are asked to
answer a series of questions concerning the decision they are struggling with. Users are presented
with a wide range of questions to choose from (see Supplementary Appendix A for the full set
of questions offered) or invited to create their own question. One choice (e.g. “go on a diet”) is
assigned to heads and the other choice (in this case “don’t go on a diet”) is assigned to tails. The
outcome of the coin toss is randomized and the user is shown the outcome of the coin toss. The
coin flippers are then re-surveyed two and six months after the initial coin toss. Additionally, prior
to the randomization, coin flippers are encouraged to identify a third party (a friend or family
member) to verify their outcomes. The third parties are also surveyed two and six months after
the coin toss.
While it might seem implausible that anyone would come to such a website and flip a coin,
much less follow the dictate of the coin toss, the results obtained speak to the contrary. In the year
of data collection, over 20,000 coins were flipped. A number of results emerge from the analysis.
First, two months into the study participants show a bias towards the status quo, in the sense
that people report making a change less frequently than they predicted they would before the coin
toss. Six months after the coin toss, however, this bias is gone.
Second, those who report making a change in follow-up surveys are substantially happier
than those who do not make a change, and they are more likely to say they would make the same
decision if they were to choose again. This is true for virtually every question asked both two
and six months later. This correlation does not, of course, necessarily imply causality. Those who
make a change differ from those who do not make a change on many dimensions.
Third, the outcome of the coin toss appears to influence the actions taken. Those who flipped
heads were approximately 25% more likely to report making a change than those who got tails.
The coin toss had a roughly equal impact on decisions across the entire range of self-stated ex ante
likelihoods of making a change (i.e. the coin toss matters whether before the toss the coin-flipper
says he/she has a 20%, 50%, or 90% likelihood of making the change). The coin toss was roughly
equally influential on men and women, the old and the young, and across income levels. The coin
minor decisions [e.g. what quality of baseball card to offer (List, 2002), whether to respond to a solicitation letter from a
charity (Falk, 2007), and when to make mail-order catalogue purchases (Anderson and Simester, 2003)].
2. To answer questions like that, previous research has typically had to rely on correlational studies (e.g.
Kalmijn et al., 2009; Pedersen and Schmidt, 2014) or natural experimental variation (e.g. Gruber and Mullainathan, 2005;
Meier and Stutzer, 2007), with the usual challenges to causal inference.
Downloaded from https://academic.oup.com/restud/advance-article-abstract/doi/10.1093/restud/rdaa016/5834495 by guest on 31 May 2020
LEVITT HEADS OR TAILS 3
toss, not surprisingly, had the biggest impact on relatively unimportant decisions like whether or
not to go on a diet, but also influenced much more important choices like job quitting and ending
relationships. The coin toss only influenced decisions made within the first two months of the
coin toss; later changes were unrelated to the outcome of the toss.
Fourth, when it comes to “important” decisions (e.g. job quitting, separating from your
husband or wife), making a change appears to be not only correlated with increased self-reported
happiness, but also causally related, especially six months after the coin toss.
3
Those who were
instructed by the coin toss to make a change were both more likely to make the change (as
noted above) and, on average, report greater happiness on the follow-up surveys. This finding
is inconsistent with expected utility theory; those who are on the margin should, on average,
be equally well off regardless of the decision they make. This result provides strong empirical
support for the notion of a status quo bias (Samuelson and Zeckhauser, 1998; Kahneman et al.,
1991). There is suggestive evidence that the coin toss outcome on “less important” decisions (e.g.
going on a diet, dying one’s hair, quitting a bad habit) influences future happiness in a similar,
but more muted, fashion.
Fifth, for all decisions—not just the most important ones—there appears to be a causal impact
of making a change on how satisfied the subject is ex post with the decision. Those who were
instructed to make a change by the coin toss are substantially more likely to report that they made
the correct decision and that they would make the same decision again if given the chance.
All of these results are subject to the important caveats related to using self-reported happiness
as a proxy for utility, a research subject pool that is far from representative, potential sample
selection in which coin flippers complete the surveys, and responses that might not be truthful.
I consider a wide range of possible sources of bias and where feasible explore these biases
empirically, concluding that it is likely that the first-stage estimates (i.e. the effect of the coin toss
on decisions made) represent an upper bound. There is less reason to believe, however, that there
are strong biases in the 2SLS estimates (i.e. the causal impact of the decision on self-reported
happiness).
The structure of the remainder of the article is as follows. Section 2 describes in greater
detail the experiment and how it was carried out. Section 3 reports the results of the experiment.
Section 4 explores how a variety of potential biases might influence the inferences drawn from the
study and also considers how likely those biases are to be important. Because this study differs
in substantial ways from standard experimental interventions by economists, the issues of bias
that arise are not the typical ones economists are used to thinking about. Section 5 concludes.
2. EXPERIMENTAL DESIGN
The experiment was carried out online at the website www.FreakonomicsExperiments.com.
4
Users who arrived at the site were greeted with the home page shown in Figure 1, which offered
3. Richard Easterlin was one of the first economists to be widely recognized for work with self-reported happiness
data, and since his contribution in 1974 on the link between income and subjective happiness many others have made use
of such data. Dolan et al. (2008) and Frey and Stutzer (2002) provide overviews of the use of self-reported happiness data
in the economics literature. Additional applications of happiness data in the field are outlined by Di Tella and Macculloch
(2006) who conclude that, treated with caution, the data have the potential to add value to empirical work. Researchers
differ in their level of optimism regarding the validity of such data—Kahneman and Krueger (2006) note that the cleanest
use of self-reported happiness data would “avoid effects of judgment and of memory as much as possible” but acknowledge
that subject to these limitations such data can add important contributions to the field, while Bertrand and Mullainathan
(2001) offer skepticism in noting that the use of a dependent variable that relies on self-reported happiness data can be
problematic because “the measurement error appears to correlate with a large set of characteristics and behaviours.”
4. For a further description of the experiment and preliminary results, written for a popular audience, see Dubner
and Levitt (2014).
Downloaded from https://academic.oup.com/restud/advance-article-abstract/doi/10.1093/restud/rdaa016/5834495 by guest on 31 May 2020
4 REVIEW OF ECONOMIC STUDIES
Figure 1
Website home page.
to help people make decisions through the use of a coin flip. Those individuals who clicked “Learn
More” saw the screen-shot presented in Figure 2. If they proceeded further, they were shown a
menu of life decisions over which to flip a coin from which they could choose; they were also
given the option of designing their own customized question. After selecting a question relevant
to their particular dilemma, subjects filled out a short survey that collected basic demographic
data, asked them to rate their current level of happiness, probed them about the decision they were
having trouble making, and gave them the opportunity to identify a third party, typically a friend
or family member, who could be surveyed in the future regarding their decision.
5
Approximately
30% of subjects provided the name and email address of a third party. This sub-sample of the data
is of particular interest for two reasons. First, naming a third party may signal greater commitment
to following the coin toss. Second, the existence of a third party provides an independent source
of information to verify later participant responses, as well as a source when the subject fails to
respond to follow-up surveys.
The participants were then led to a page where a simulated coin tied to a randomizing algorithm
was flipped and came up either heads or tails.
6
Subjects were reminded of what action the coin
toss directed them to take, and if the coin toss said to make a change, they were encouraged to
5. Users were also shown, at random, a fact relevant to the decision they were about to make. For instance, those
pondering whether to quit their job were told either “The number of job openings is on the rise—up by nearly 70%
since 2009” or “Workers who dislike their jobs report lower levels of wellbeing than the unemployed. In fact, 81% of
the unemployed report that they are happy every day compared to only 69% of the unhappily employed.” There are no
statistically significant differences in actions associated with having seen different facts.
6. Before the coin toss took place, subjects were asked how likely they were to make the change. If subjects
indicated that they were very likely or very unlikely to make a change, they were taken to a page telling them that it
seemed like they had already made up their mind. Those subjects then had the option of proceeding to the coin toss or
exiting. All users were given the choice of having their outcome determined by a single coin toss, or could opt for a “best
two out of three.” Approximately 56% of users chose the “two out of three” option. In terms of subsequent behavior,
Downloaded from https://academic.oup.com/restud/advance-article-abstract/doi/10.1093/restud/rdaa016/5834495 by guest on 31 May 2020
LEVITT HEADS OR TAILS 5
Figure 2
What potential study participants saw when they clicked “Learn More”.
make that change within the next two months. In those cases where the coin toss said don’t make
a change, the subjects were told to maintain the status quo for at least the next two months (e.g.
if the coin toss said not to quit one’s job, the subjects were asked to remain at the job for at least
two months). In most, but not all cases, heads was associated with making a change and tails was
associated with maintaining the status quo. For simplicity in exposition, I refer to heads in what
follows as meaning that the coin toss recommended a change.
Subjects were aware that they were part of an experiment and were required to explicitly give
their informed consent. Both the subjects and the third parties provided by the subjects were
then surveyed two and six months after the coin toss. Survey reminders were sent via email
and included a link to an online survey site where the follow-up surveys were done. In order to
encourage survey completion, those who filled out the surveys were provided with small gifts that
took the form of exclusive content from Freakonomics podcasts. It should be noted, however, that I
intentionally made it difficult for subjects to determine the precise objective of the study. Subjects
were told that their participation would “help us gain important insights into decision-making.”
The initial survey, prior to the coin toss, asked many questions about motivations and feelings
surrounding the decision. The follow-up surveys also asked a number of questions unrelated to
the actual purpose of the study.
The website FreakonomicsExperiments.com was launched on 23 January 2013. Recruiting
was done through a variety of online and traditional media avenues including reddit.com, the
Freakonomics podcast, the Freakonomics blog, Marginal Revolution, and articles published in
The Financial Times and Forbes. Data collection at the site remained active for roughly a year,
after which a scaled down version of the site remained operational, but all survey activity ended.
there are no clear differences between those who went for the single coin versus best of three option. In what follows, I
use the shorthand of a coin toss to refer to both of these options.
Downloaded from https://academic.oup.com/restud/advance-article-abstract/doi/10.1093/restud/rdaa016/5834495 by guest on 31 May 2020
6 REVIEW OF ECONOMIC STUDIES
TABLE 1
Question attributes
Number Important? Choice between action
Question of tosses question? and Status Quo?
Should I quit my job 2,186 Yes Yes