Goodstein on Refereeing

`Every Scientist I know has stories of being treated unfairly by anonymous referees.'
The real problem in science today is that the institutions of science and the way we operate the scientific enterprise are all based on entities that evolved and took on their present form during the era of exponential growth. They are suited to exponential growth, but not to the kind of world we face in the future.

Let me give you two simple examples. The first example is peer review, in which articles for prestigious journals, as well as proposals for funding from foundations or government agencies, are sent out to anonymous referees in the field to review them. Now, peer review is a very good way of separating valid science from nonsense because the referees always know what is going on in a field, know the conventions and the directions of thought in their field, and can recognize immediately if something is not following that line of thought. But it is a very poor way to adjudicate an intense competition for scarce resources, such as the funds of government agencies or pages in prestigious journals.

The reason that peer review functions poorly in conditions of scarce resources is perfectly obvious to anybody who stands outside of the system: Referees are chosen because they are among the few experts in the field at hand, so they are also competitors for the same resources. They have a conflict of interest. Their anonymity is protected by the editors or contract officers for whom they are doing professional service. And it would be very difficult for them to overcome that protected anonymity and still do a good job; they must have very high ethical standards to referee fairly. The fact is that most scientists are people with very high integrity and do their job well.

However, every scientist I know has stories of being treated unfairly by anonymous referees when they themselves were authors. That kind of experience erodes your ethical standards, until you get to the point where you say, ``Why should I be the only good guy?'' When that happens, the whole system crashes, which places peer review, I believe in severe difficulties...

The second example is one that you might not have thought of. The universities are both the intellectual and the economic entrepreneurs of science. Universities go out and borrow money to build laboratory buildings and buy the latest state-of-the-art equipment, then invest huge amounts hiring tenured professors to fill those laboratories. When they do that they are making a business decision and, although universities are not businesses, they must in some sense be run as businesses because somebody has to pay the bills. They are making business decisions that assume that those professors are going to bring in enough money in grants and contracts in the future to pay for the investment in the laboratories, buildings, and the professors.

In the era of exponential growth, that was a winning business strategy. Today, it is a disastrous business strategy. Either the universities will learn to stop building those laboratories or they will go belly up. Either way, we will not have the entrepreneurs in the next generation to build the next generation of scientific laboratories and instruments.

The bottom-line issue is whether science can survive constraint or not. In the past, science was a competition against nature: Could we be clever enough to overcome nature, to find out her secrets by doing successful science? That is no longer the case. Science today is a competition for resources, which makes it an economic, rather than an intellectual, competition. There is no guarantee at all that science will flourish or even survive under these circumstances.

The marketplace, people say, will take care of everything, and, in truth, the marketplace does work its wonders. The marketplace, after all, is responsible in some sense for the fact that we stopped producing exponentially increasing numbers of Ph.D.'s in physics after 1970. But the marketplace does not care about individuals and does not care at all whether science survives...

Most people in the academic world who have heard of my arguments (a remarkable large number) assume that I am arguing for some sort of ``inflexible birth control,'' meaning to stop the production of Ph.D.'s. I do not believe and I cannot believe that the solution to this problem is less education. I do believe that the solution to this problem is more education in science, but it has to be of a different nature.

I do not know whether or not there is an excess of people with Ph.D.'s trained to do research, but, at any rate, these people are very clever and tend to get jobs somewhere even if not doing research. They tend not to be unemployed (even though some undoubtedly are). In contrast, there are 20,000 high schools in the United States that do not have a single trained physics teacher. It seems to me that we have not a surplus of scientifically trained people in this country, but a severe shortage of such people.

The problem is that those who have been trained in science have been given the wrong expectations. Either that or perhaps we have arranged society so badly that people who are willing to put themselves through the long period of time it takes to get a scientific education are then unwilling to become high-school teachers. Can you imagine a society in which teaching in high school were a respected, prestigious-enough occupation that people would be willing to undergo a long, serious training in science in order to teach in high school? If you can imagine that, you are imagining the solution to all of the problems that I have been talking about.

David Goodstein, vice provost and professor of physics and applied physics at the California Institute of Technology, at a Washington roundtable on science and public policy sponsored by the George C. Marshall Institute

Copyright (©) 1996 by The Chronicle of Higher Education, Inc.
Title: `Quotable' (page B5)
Published: 96/08/02

Your comments and suggestions are appreciated.
[OSU Physics] [Ohio State University]
Edited by: [September 1996]