Is Peer Review Forcing Academics to Become Prostitutes?

Prostitute-cartoon-300x169

Recently I came across a fantastic blog entitled Rejection Letters of the Philosophers. It is a satirical blog in which famous philosophers are imagined to have submitted their manuscripts to their peers via the contemporary academic peer-review system. The humour lies in writing the rejection letter from a completely clueless and bigoted referee who merely want to defend already-existing ideas. (Sorry to be so analytical, I know that to explain the joke is to kill it, but I cannot assume that all my readers will go and read the blog).

After I read the posts, I sent in one of my own. In it I imagine a rejection of George Berkeley’s path-breaking, though largely ignored, essay De Motu by a Newtonian living around the same time that Berkeley published the essay. It can be read here.

I discussed the actual issue underlying the blog with the owner at some length via email. He directed me to this fascinating paper entitled Publishing as Prostitution? — Choosing Between One’s Own Ideas and Academic Success. It is written by an economist called Bruno Frey and, although it employs a utility-based analysis that I find reductionist, it nevertheless raises some very interesting points that I believe are worthy of further consideration.

Although it might seem strange to readers of this blog that a utility-based analysis might yield anything of interest I should say clearly that I do not think this is actually the case; rather I think that the author has cleverly bent utility-analysis to suit his own purpose, which is to criticise the Groupthink dynamic generated by the contemporary peer-review process. All the salient points that the author makes can be made just as clearly — and perhaps more forcefully — outside of a utility framework. That said, let us now turn to the substance of the paper.

The author identifies the fact that the peer-review process can often give rise to what he calls “intellectual prostitution”. The idea lying behind this is that the author trying to publish in a journal sells themselves by bending and altering their ideas to fit those that are considered to be “good” by their peers who are refereeing their paper. In an ideal world it would be expected that referees and editors, provided they see that the paper conforms to basic academic standards (clear argument, good format, references etc.), would then engage in “light touch” editing; that is, they would suggest possible improvements; where clarity might be enhanced; or suggest slight changes in layout. This, for example, is how the sub-editing process in a newspaper works.

Frey notes, however, that this is rarely the case. Rather the referee process is used to try to push group opinion on the author. Frey writes:

All authors would like to receive referee reports helping them to improve their paper. Alas, this is rarely the case. Normally, the referees want to see substantial changes basically altering the paper. Often, an almost completely new paper is demanded. At the very least, the author is asked to write things he or she would not otherwise have written. The more fundamental and numerous the changes demanded by the referees are, the less it pays to submit a paper and to engage in an academic career. (p208)

Frey is quick to note that this should not be considered a bad thing per se. As he notes with regards his analogy, it is possible to take the view that prostitution is a voluntary market act taken in line with a given monetary incentive that does no harm to anyone else and so has no externalities (whether one agrees with this evaluation matters little to the argument). So why then can we not view intellectual prostitution in the same manner? Frey writes,

Scholars are seen as performing a similar activity to artists, in particular painters who, since the Renaissance, are expected to express their own beliefs and convictions – which led to an explosion of creativity in the arts. The almost dictatorial demands advanced by the referees are difficult, or even impossible, to reconcile with authors wanting to publish their own ideas in economics journals. (pp206-207)

Frankly I think that in this passage Frey is bending his utility analysis until it breaks. He is clearly here making a moral or aesthetic judgment as to what “good” academic work should be. He cannot justify this on the basis of cold utility analysis. I could point out the flaws here in more depth but that is not my goal as I think that, while the form that the paper adopts is disingenuous, the substantive issues it raises are very important and the author’s contribution extremely interesting.

Frey goes on to identify what he believes to be the key reason that peer-review may lead to intellectual prostitution. He writes,

A useful starting point for a rational choice theory of referees’ and editors’ behavior is to acknowledge the difference between the two groups of actors on property rights to journals. Anonymous referees have no property rights to the journal they advise. They are not concerned with the effect their advice has on the journal. The absence of property rights must be expected to lead to shirking. The interests of the journal and the referees are not aligned… Many referees will be tempted to judge papers according to whether their own contributions are sufficiently appreciated and their own publications quoted. They carry, for instance, no costs when they advise rejection of a paper they dislike (e.g., because it criticizes their own work), even if they expect that it would be beneficial for economics as a discipline. (pp208-209)

I think that Frey is absolutely correct here. Hidden under the cloak of anonymity referees need not be neutral at all. Indeed, they can be as factional, subjective, authoritarian and nasty as they want and it will not do any damage to their reputation. This is, of course, precisely what often happens. If one is handed a paper to referee and one doesn’t like the ideas contained because they conflict with your own, you have no disincentive not to act like a child and demand that the author of the paper conform to your own narcissistic view of the world.

I say that this is true because, having trained as a sub-editor, I know that this habit has to beaten out of people. When you start sub-editing newspaper copy the temptation to insert into it your own views is enormous. You have been imbued with power over another person — over their thoughts, one might say — and while all power corrupts, absolute power corrupts absolutely. This tendency persists even when you have recognised that this impulse is authoritarian, immature and tasteless. To take a truly neutral view of the material is extremely difficult. I believe that this is almost impossible for the average anonymous referee who has not gone through any such training process.

So, what does Frey recommend? Well, he says that we should recognise basic property rights. The owner of a journal is the editor and that editor’s reputation — unlike that of his/her referees’ — hangs on his/her journal. If the editor is forced to make the decision alone his/her reputation will be damaged if he/she is seen to be persistently engaged in bias. Personally I think that there is a lot of wisdom in this proposal. It is certainly better than the system we have now which is, as the author says, driving creative young academics out of academia and amassing power to an older generation unwilling or unable to change their minds on certain issues.

Finally I should say, given that this is an economics blog, that this does not just apply to the mainstream economic journals. I have not had a great deal of experience with the heterodox journals, but it should be noted that the two heterodox economists whose work is currently in vogue — that is, Hyman Minsky and Wynne Godley — largely did not publish in these journals. It should also be noted that the force that has popularised heterodox economics more than any other — I think of Modern Monetary Theory (MMT) — has done so outside of established journals. This suggests, to me at least, that there may be something rotten in the state of Denmark, as it were. The market for ideas is quite manifestly in disagreement with the contemporary journals.

Addendum: I should note that while I believe I have brought out the salient points of the paper above, it is nevertheless worth reading as there is much else of worth in there. At some points the paper almost seems like a well-written satire; especially when the author discusses “born intellectual prostitutes” and “learned intellectual prostitutes” on page 211.

Advertisements

About pilkingtonphil

Philip Pilkington is a London-based economist and member of the Political Economy Research Group (PERG) at Kingston University. You can follow him on Twitter at @pilkingtonphil.
This entry was posted in Politics. Bookmark the permalink.

15 Responses to Is Peer Review Forcing Academics to Become Prostitutes?

  1. Jan says:

    Great piece Phil!So Hume, Hobbes, Machiavelli,Schopenhauer et al was not good enough for the peers??Amazing!It don´t surprice me though.Well Uppsala University rejected Foucault´s The History of Madness. as doctoral dissertation, but there are many exampels still today of course.I remember one of Swedens greatest poets got refused for his book with the name “Why do i live?” In the letter it stood – “The only reason for that, is that you send your manuscript by mail and not delivered it personally!”. They could be rather brutal those “peers”!

    • Well, I don’t know how those guys were treated. But they lived in a different time with a largely liberal publication regime and a different manner of disseminating texts. Certainly they were never published and disseminated through the old Schools.

      Today we are in a sort of neo-Scholastic mode where the Academy — in the form of its minions — exercise control over publication. The same dynamic of suppression of new ideas is at work too, I think.

  2. NeilW says:

    I think the peer review system has completely broken down and is completely incompatible with the modern publishing world – where getting something out there is trivial and space is no longer at a premium.

    For me the modern approach would be to filter papers based upon an XML stylesheet – so that they are semantically rich and can be handled effectively by the Internet’s search engines.

    The standard format paper can then be generated simply by applying a stylesheet to the raw information.

    The biggest problem with academic papers is that at best they are in PDF format – often badly formatted as an image.

    • This sounds like an interesting approach. Have you ever discussed with the editors of the online heterodox journals?

      • NeilW says:

        The problem is that they don’t want the system to be ‘open source’ – since publishers are all about controlling the channel.

        The whole fundamental design of the web is about publishing documents in the open for all to view and find – in the classic academic manner.

        However recently it appears that the commercial copyright system and intellectual property mania has taken over academia.

        And that’s because Universities are now about bums on seats and serving ‘customers’ – rather than just being there to advance understanding.

        Neo-liberalism’s insidious grip on the world is slowly suffocating the one alternative model that had been proven to work.

  3. spassapparat says:

    I can agree with most of the article, but there are a couple of problems in my opinion in the last few paragraphs:

    – First off, I do not really think that the editor or the board of editors of a journal has any more of an incentive to allow more open views than the peer reviewers do. Imagine the AER publishing a paper on Marxist class analysis. Don’t you think that this would significantly hurt instead of bolster the reputation of the AER?
    The groupthink pressure is similarly strong for editors as it is for peer reviewers. The only thing they have to fear but peer reviewers don’t is that if a paper is rejected on grounds of personal dislike but subsequently published in another journal, this might hurt the reputation of the journal in question if a) this paper is so good that it should have been published by the economics journal in question and b) the rejected scholar makes the information that his paper has been rejected by the journal in question publicly available.

    Hence it is questionable whether this would lead to any changes. It crucially depends on the likelihood of the paper getting published in any respected journals. This makes it clear to me that following such a change sligthly more innovative mainstream papers might be published in mainstream journals, but it definitely would not lead to a truly pluralist journal culture.

    – Secondly, while I am fairly certain that you are right that the same problems with peer review are present in heterodox journals as well, I find the evidence you present pretty weak. Why should Minsky and Godley publish in mainstream journals based on the fact that heterodox journals have problems with their peer review process, if mainstream journals very much have the same problems? I’d argue that it has more to do with respectability.

    – Lastly, there has been substantial publishing of MMT ideas in economic journals before the NEP or other MMT blogs have been started. The fact that MMT got popularized over the internet is, in my opinion, not a reflection of heterodox journals not being open to MMT ideas. It reminds me much more of an article by Stockhammer and Ramskogler a couple of years ago on ‘Post-Keynesian economics – how to move forward’, where one of their points was that if you have great ideas, but the important journals won’t publish you because they want to cling on to their bad ones, then you should be politically relevant and engage with the public. People are smart enough to distinguish decent arguments from bad ones if they are presented in a way as to make them amenable to the public, and that’s what explains MMT’s success in my opinion.

    • (1) It’s not really about that. A Marxist journal can publish the Marxist stuff. Whatever.

      (2) Minsky and Godley had very obscure publication records. The only people that really took them on were the Levy Institute (who have a fantastic track record). I’m hinting at something here that I’ve heard second and third hand that I don’t really want to get into because I don’t know if its based on fact. I don’t have much experience but from what I hear the heterodox journals do have the same problems.

      (3) My impression is that PKs tended to ignore MMT until it broke online. That’s just my impression. The fact that it broke online first suggests something to me about the structure of the journals. It should have broke in the journals first and then online if the journals are relevant.

      • spassapparat says:

        (1) Well it matters because the journals have different impact factors and the AER papers have a much higher chance of becoming relevant in the public discourse as well through newspapers or the like. But ok, this is a whole different animal.

        I still think the main part of my argument still stands. Frey’s proposed solution won’t change anything. It might even worsen the problem since who get’s to publish is decided by a smaller amount of people.

        (2) I understand. I guess one would have to ask people who actually worked with Godley or Minsky to find out why their publication records look as they do.

        (3) Well that’s a question of definition. what would you consider to be ‘breaking’, and what would you consider to be MMT?
        I’d say that people like Wray diverged from traditional PK views in the 1990s already but still he got published quite frequently in the JPKE.

      • (1) I agree. But that’s really a whole separate issue. The article specifically raises the point about the peer review system.

        I do not think that it would worsen the problem. The problem of mainstream journals is as bad as it gets. It’s a case of tending to our own garden here.

        (2) I get a strong impression that it was due to some of the problems laid out above.

        (3) Wray was already established as a leading theorist in the endogenous money community, so I don’t think that counts. I’m just saying that I get a strong impression that the PK community largely ignored it until it hit the net. They were really behind the curve on that front.

        Again, anything I say — which I’m not going to anyway because I have no desire to start naming names and picking fights based on second or third hand information — is not going to be very substantive. But everything I hear leads me to believe that these problems exist. And I agree with Frey that they exist, at least in part, to the peer review system. It gives anonymous referees too much sway in pushing the party line or trying to prove how “clever” they are by picking away at an article needlessly.

  4. ivansml says:

    Yep, Frey knows a thing or two about publishing ethics… http://economiclogic.blogspot.cz/2011/09/bruno-frey-bubble.html

    • First of all, this has nothing to do with ethics. This has to do with the validity of the peer-review process. If you want to defend the latter do so directly, not by pointing to wayward critics.

      Secondly, Frey’s actions could easily be read as an attempt to dupe the very system that he criticises. Perhaps it is poor advice for students. But that does not make it ethically inconsistent. It appears that he thinks that the drive to publication in peer-review journals is a waste of time and then tried to figure out a way to stop wasting his time. Again, not the best thing to pull students into. But ethically hardly internally inconsistent.

      • ivansml says:

        I wasn’t defending publishing system, which I’m sure is far from perfect. It’s just… let’s say amusing to hear these criticisms from somebody like Frey, who has played the system pretty well so far (well, until he got caught).

      • I see no irony. It seems totally consistent to me.

        (1) Publish article condemning the system as being aimless and stupid.

        (2) Try to figure out a way to game the system so that you can do real work.

        (3) Get caught for something that isn’t even that bad (self-plagiarism) and not get in that much trouble but have those that you’ve pissed off jump all over you.

  5. Pingback: La prostitución intelectual que impone el sistema de revisión por pares | Francis (th)E mule Science's News

  6. Reblogged this on glasgow greek boots and commented:
    I don’t think that accademic prostitution stops here. What is called favouratism, now becomes networking.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s