FEBRUARY 2017
|
PHYSICS TODAY 49
awardee Hutchinson sued him for libel. Proxmire eventually
made a public apology to the psychologist and omi!ed indi-
vidual names from future Golden Fleece press releases.
Although most of the criticisms were leveled at the social
sciences, scientists from across disciplines followed the contro-
versy. P
HYSICS TODAY reported closely on the hearings and on
NSF’s policy changes. Editor-in-chief Harold Davis argued in
an editorial that the hearings demonstrated “that peer review
is by far the best means we have for deciding how funding
should be distributed in a given area.” (See P
HYSICS TODAY, Sep-
tember 1975, page 96.) In the same editorial, Davis went on to
announce that P
HYSICS TODAY would be sending complimen-
tary issues to every member of Congress to illuminate the inner
workings of the scientific community. As Davis put it, “In an
age in which the issues of society cannot avoid being ever more
closely involved with science and technology we are going to
need more peer review, not less.”
The 1960s and 1970s seem to have been a crucial period of
transition for ideas about peer review. In the mid 20th century,
external refereeing was simply one of several methods a jour-
nal or grant-issuing organization could use to choose which
submissions to accept or reject. By the end of the Cold War,
peer review was a prerequisite for scientific respectability.
The NSF controversy strongly suggests that one reason for
the increased emphasis on peer review, at least in the US, was
a shi"ing relationship between scientists and the public during
the Cold War. Spending on both basic and applied research
had increased dramatically in the 1950s and 1960s—but when
doubts began to creep in about the public value of the work
that money had funded, scientists were faced with the prospect
of losing both public trust and access to research funding.
Legislators wanted publicly funded science to be accountable;
scientists wanted decisions about science to be le" in expert
hands. Trusting peer review to ensure that only the best and
most essential science received funding seemed a way to split
the difference.
Peer review in crisis?
Today peer review is an expected part of publishing any sci-
entific article or obtaining grants. However, few would argue
that it is a perfect process. Many observers have lamented
that fraudulent or flawed results still reach the pages of peer-
reviewed journals. Others complain that the peer review system
favors established ideas and institutions and stifles scientific
innovation.
In 2014 Michael Eisen, a cofounder of the publisher Public
Library of Science (PLOS), told the Wall Street Journal that sci-
entists and nonscientists need to discard the notion “that peer
review of any kind at any journal means that a work of science
is correct. What it means is that a few (1–4) people read it over
and didn’t see any major problems.”
11
Another drawback with the current peer review system is
that the work reviewers put in generally does not count toward
tenure or promotion. Overburdened scientists face li!le incen-
tive to write long, careful, and detailed reports that go beyond
discharging their minimum duty as good scientific citizens.
The shi" to online publication and reading seems to suggest
alternative methods for ve!ing articles, such as allowing scien-
tists to post comments about those they read. Physicists have
long relied on the non-peer-reviewed arXiv.org to find the lat-
est publications in their field, although readers may regard a
paper posted to the arXiv but never published in a journal as
somewhat questionable.
Other journals have been experimenting with slightly al-
tered peer review systems. PLOS One, a well-known open ac-
cess journal, instructs its referees to judge only the quality of the
science in the paper, not the work’s perceived importance or
impact. The reasoning behind PLOS One’s policy is that work-
ing scientists will determine which papers are most important
a"er publication. Another journal, eLife, puts referees and edi-
tors in communication with each other to arrive at a single joint
decision on a paper’s future, rather than sending authors mul-
tiple reports that might disagree wildly with one another.
As the scientific community considers peer review’s future,
it may be instructive to consider its past. We o"en speak of ref-
ereeing as something that has been a stable and unchanging
part of science ever since the age of Isaac Newton, but peer re-
view’s story is both shorter and more complex than we o"en
assume. It is also li!ered with criticism. As early as 1845, the
scientific referee was described as “full of envy, hatred, malice,
and all uncharitableness.”
3
Complaints about reviewer useless-
ness and bias, in other words, are hardly new.
It also seems significant that refereeing procedures were not
initially developed to detect fraud or to ensure the accuracy of
scientific claims. Whewell thought referee reports would spur
scientific discussion, and scientific societies adopted refereeing
to ensure that nothing obviously embarrassing reached print.
Authors, not referees, were responsible for the contents of their
papers. It was not until the 20th century that anyone thought
a referee should be responsible for the quality of the scientific
literature, and not until the Cold War that something had to be
peer-reviewed to be seen as scientifically legitimate.
Peer review’s role in the scientific community has never
been static. Its form and purpose have been shaped and re-
shaped according to what scientists needed from the practice—
whether it was credibility for a scientific society, improvements
in the scientific literature, or assurances to public funders that
their money was being spent responsibly. If scientists are to
tranform peer review’s future, they must consider what pur-
pose they want it to serve—and whether that purpose can in-
deed be fulfilled by reports from two or more referees.
REFERENCES
1. H. Zuckerman, R. Merton, Minerva 9, 66 (1971).
2. A. Fyfe, J. McDougall-Waters, N. Moxham, Notes Rec.: R. Soc. J.
Hist. Sci. 69, 227 (2015).
3. A. Csiszar, Nature 532, 306 (2016).
4. M. Baldwin, in The Age of Scientific Naturalism: Tyndall and His
Contemporaries, B. Lightman, M. Reidy, eds., Pickering and Chatto
(2014), p. 171.
5. I. Clarke, Isis 106, 70 (2015).
6. R. Lalli, Notes Rec.: R. Soc. J. Hist. Sci. 70, 151 (2016).
7. R. Kohler, Partners in Science: Foundations and Natural Scientists,
1900–1945, U. Chicago Press (1991), p. 68.
8. I. H. Page, J. Am. Med. Assoc. 225, 1240 (1973).
9. D. Kaiser, Nature 505, 153 (2014).
10. J. Conlan to H. G. Stever, 15 May 1975, in National Science Foun-
dation Peer Review, Special Oversight Hearings, US House of
Representatives, p. 21.
11. M. Eisen, quoted in H. Campbell, Wall Street Journal, 13 July 2014,
http://www.wsj.com/articles/hank-campbell-the-corruption-of
-peer-review-is-harming-scientific-credibility-1405290747.
PT