Publication credentials are known to heavily impact academic career progression. In this age of rankings and metrics, prestige matters. It would therefore be dispiriting to academics to find that journals perpetuate systemic biases that work to the detriment of colleagues at less prestigious institutions. But in US legal academia, it has long been suspected that journal editors (most of whom are students) are affected by “letterhead bias”, where publication decisions are unduly influenced by the institutional affiliation of submitting authors.
I decided to investigate the existence of this phenomenon, and my research findings, recently published in the Harvard Journal of Law & Technology, provide strong statistical evidence for the existence of letterhead bias.
The study was the largest audit of US law reviews to be conducted to date, covering more than 4,500 articles and almost 6,000 authors. It analysed the demographics of authors published in the top 50 (“T50”) US law reviews, as ranked by the Washington and Lee Law Journal Rankings, across the five calendar year period from 2014-2018 inclusive.
With editorial boards tending to change on an annual basis, this offered a wide enough sample to mitigate the idiosyncrasies of a single editorial board. A methodology was devised to measure the median institutional prestige of authors published in each T50 journal, and by applying linear regression analysis a clear correlation was established between the ranking of a journal and the institutional prestige of its authors.
The standard objection – which was adequately discredited in the paper – is that better scholars (lazily defined as scholars affiliated to more prestigious universities) will write better papers and thus publish in better journals. There is a great deal wrong with that statement, from its question-begging to its inherent elitism.
In addition, other findings in my study further strengthened the evidential basis for letterhead bias. I measured what I called “self-publication”, where a journal publishes the paper of an author affiliated to the same institution as that of the journal. Aside from the extensive conflicts of interest this generates for student editors, the figures were startling. The average T50 journal had a self-publication rate of 7.7 per cent, but linear regression analysis showed that higher ranked journals tended to have higher self-publication rates.
In some cases, these rates were astonishingly high, peaking at 24 per cent for Virginia Law Review, followed by 20.4 per cent for New York University Law Review, and 20 per cent for Harvard Law Review.
When almost one in four of a journal’s authors are affiliated to the same institution as the journal, it is difficult to explain this on any other basis than letterhead bias. This will confirm the suspicions of many legal academics across the US and give them further cause to regret the current law review system. These are, after all, highly coveted publication spots.
The good news is that not all T50 journals had high self-publication rates. The journals with the lowest self-publication rates were Boston College Law Review with a rate of just 0.7 per cent, William & Mary Law Review with a rate of 0.8 per cent, and Connecticut Law Review with a rate of 1.2 per cent.
Various student editors also communicated to me their concerns about publishing their own faculty’s work. Commendable as that is, it does not alleviate the broader concern about letterhead bias: student editors are put in such a difficult position in being required to appraise the work of experienced scholars that they will almost inevitably turn to proxies for article quality.
The institutional letterhead of an author can then be used as a staple yardstick, with a prestigious affiliation bestowing a presumption of quality, and a more humble affiliation lumbering the paper with a presumption of inferiority.
As journals seek to boost or maintain their own prestige, they are also incentivised to publish the work of authors at more prestigious universities; thus the journal and the authors feed off, and perpetuate, each other’s prestige. The author at a law school outside the top tier (and, as I discuss in the paper, the overseas author) therefore begins at a considerable disadvantage, regardless of paper quality.
This is not to say that such biases do not manifest in the world of peer-reviewed journals, but the US law review system can be considerably improved by two reforms. First, there should be a rule against journals publishing their own faculty’s work, significantly reducing conflicts of interest and the potential for cronyism.
Second, law reviews should institute a comprehensive policy of blind manuscript review, so that author identity is substantially removed as a potential factor in making publication offers.
These initiatives would not be a panacea for the law review system’s deficiencies, but they would introduce a much more level playing field on which all authors compete. One of the main obstacles is that the scholars who benefit most from the existing system have the least incentive to change it, and the most incentive to obstruct change. That is why a data- and evidence-based approach to an exposure of the current system’s shortcomings is preferable to rumours and anecdotes – it makes the existing system palpably less defensible – and it is what inspired my statistical investigation of letterhead bias.
It is my hope that these data will be used to more firmly advocate for change within the law review system, to push for its transition from an elitist closed shop to a fairer and more egalitarian determinant of academic careers.
Stephen Thomson is associate professor of law at City University of Hong Kong. His findings were published as “Letterhead Bias and the Demographics of Elite Journal Publications” in the Harvard Journal of Law & Technology.
