Refereed Journal Articles vs. the Entertainment Industry

I am occasionally asked to explain the difference between the peer-reviewed journal literature and, for lack of a better descriptor, the rest of the publications in the world. The latter is a big bucket, so I will try to trim it a bit, primarily by example. I’ve hinted at this topic previously.

Most people receive information primarily from corporate sources. For the relatively few people who actually read, the usual publications include newspapers and magazines such as Newsweek, Time, and People. These publications are churned out by corporations to entertain the masses and sell products. They have proven remarkably successful on both fronts.

There are many differences between these conventional magazines and refereed publications. Most notably, authors of the latter publications pay to have their work printed in outlets that are expensive to purchase. Furthermore, results are published once in the journal literature, whereas every catchy “news” item is spewed ad nauseum by the corporate media. In addition, the processes by which the two disparate publications see the light of day are quite different. I elaborate below.

Consider, by way of example, the prestigious journal Proceedings of the National Academy of Sciences (PNAS), the official scientific journal of the National Academy of Sciences (NAS). The NAS is not in the business of making money, and publishing is an expensive undertaking. As a result, as with most refereed journals, authors pay a fee to have a paper published. The fee, typically called “page charges,” usually is based upon the length of the final publication. Page charges typically run a few hundred dollars per page, although some journals charge a flat fee of a thousand dollars or so. In other words, authors of journal articles do not get paid to publish their work. Rather, they–or, more often their institution or the grants they have worked diligently to secure–bear the monetary cost of publishing their results.

Imagine a freelance journalist having to pay Time for the honor of appearing in print. Under this imaginary scenario, the world would soon be devoid of freelance journalists.

Another significant difference between the two kinds of publications is the process by which a paper appears in print. In the case of Newsweek, Time, and People, a staff writer or freelance journalist pitches an idea to an editor. Once the idea is approved, research is conducted and the story is written (it is not as if the process is as easy and fun as this cryptic description makes it seem). If the result meets journalistic standards for ethics and credibility and also has the all-important ability to attract readers and therefore advertising dollars, the paper is approved for print by an editor. The editor need not, and probably does not, know much about the topic. Contrast this approach with the customary approach taken in the arena of peer-refereed journals.

In the case of PNAS, and most other refereed journals, the idea for the final published work is vetted in the scientific community in many deliberate and time-consuming ways. The initial germ of an idea is discussed with colleagues on campus and during conferences. It is scrutinized and put to a test, the nature of which often requires years of research in the laboratory and in the field (aka the actual world). Results are evaluated with detailed measurements. Numbers are crunched using a stunningly sophisticated array of computational methods and devices. Initial results are presented to colleagues via seminars on campus and then at conferences. Every conceivable error is sought by critical audiences eager to find fault with new information. Finally, the written results are submitted to a journal for consideration. The fun ends here.

The submitted draft is read by an editor with expertise in the field. In general, and to keep this description simple, s/he has a couple options: (1) reject the paper as inappropriate for the journal or lacking in substance, or (2) accept the draft on behalf of PNAS, and send it to subject-matter specialists. Either way, a confirmatory note is sent to the author who has taken on the task as corresponding author for the paper. Obviously, rejecting the paper means the end of this story for this paper in this journal. The authors can call an end to this project, after years of scholarly work, or they can submit the paper, generally after many hours of additional scholarship, to a different journal. If the latter route is chosen, the scholarly version of “rinse and repeat” begins.

If the paper is accepted for review, then it is send to reviewers. These are the experts on this particular topic who directly compete with authors of the submitted paper for funding and prestige. Three or four of them typically are selected by the editor. Their names are not revealed to the authors of the paper under consideration. This process is called blind review. Increasingly in the social sciences, and very rarely in the natural sciences, double-blind review is used (in which author’s names are redacted by the editor before the paper is sent to reviewers). Bear in mind that all reviewers are selected because they are renowned in their field of study. They are busy. They are not paid to review the work of their colleagues. One could argue they are rewarded for providing a late, scathing review or no review at all. I have seen years-old manuscripts on the desks of former colleagues, promised for review within two weeks.

Once the editor receives detailed reviews, often after harassing unpaid, busy reviewers for several months or years, s/he is charged with making a decision that falls into one of four categories (I am lumping, not splitting): (1) reject the submission as unsuitable for this journal, with no opportunity for additional consideration, (2) reject the paper as submitted, with an opportunity for submitting it again once various criteria are met, (3) accept the paper, conditional upon responding to criticisms raised by reviewers (which could include abundant additional research), or (4) accept the paper as written (the equivalent of opening a book in a large library and having several thousand dollars fall out). In my lengthy academic career, I had each of these experiences (the latter only once, with a journal lacking stature, and I would have preferred the thousands of dollars). The middle two outcomes are most common, in large part because academics soon learn how to avoid option (1) and few are lucky and skilled enough to experience option (4). The process from the germination of a new idea to the resulting refereed publication, depending upon the required infrastructure and observations, typically requires between a year and a decade. Stunningly to me, my academic work continues to be cited, as I was reminded via email earlier this week.

There is a striking difference between conventional, corporate publications with which most people are familiar and refereed publications. The former are designed to make money for the corporations publishing them. The latter are designed to generate reliable knowledge for the informed citizenry. Edward Bernays loved the former. Only intellectually curious trackers of truth appreciate the latter.