Written evidence submitted by Professor
Howard Elcock, AcSS (PR 06)
SOME PROBLEMS WITH THE ANONYMOUS REFEREEING
SYSTEM IN THE UK SOCIAL SCIENCES
1. I am Professor (emeritus) of Government
at Northumbria University As an academic specialising in Political
Science and Public Administration of some 40 years' standing,
I have had extensive experience both of submitting articles and
research projects for refereeing. I have also acted as such a
referee myself on many occasions, especially as a member of the
Editorial Board of the Journal Public Policy & Administration.
2. Some years ago the Royal Society expressed
concern about the working of the anonymous refereeing system which
is now in general use for assessing manuscripts submitted for
publication in learned journals, as well as to vet applications
for research funding. The Royal Society appointed a working party
chaired by Lord Rees of Ludlow, to investigate this and other
problems concerned with disseminating scientific knowledge to
the general public. In the light of the Society's concern about
the refereeing system, I raised this issue with the Committee
of Academicians of the Academy of Social Sciences. The ensuing
discussion made it clear that my occasionally adverse experiences
were by no means unique. The Royal Society's working party (Royal
Society, 2006) confirmed the existence at least some of the problems
indicated in this submission. The present document is a revised
and updated version of that submission.
3. The first major problem is that unlike the
United States and possibly other countries, British communities
of scholars are usually too small for refereeing to be truly anonymous:
one Editor has said to me that "the small world we inhabit
certainly presents real difficulties in securing genuinely anonymous
refereeing, especially in some areas of specialised work".
For example the Political Studies Association, which is probably
among the larger British social science professional bodies, had
in 2006 some 1,600 members, which constitutes a considerable majority
of political scientists teaching in British universities and other
higher education institutions (see PSA, 2006). While this is perhaps
too many colleagues for any of us to be personally acquainted
with all of them and their work, this would not apply to the more
senior members who are likely to be both the more extensively
published researchers and to be asked most often to act as anonymous
referees - they are perhaps between a third and a quarter of the
total. They may well personally know most of those involved in
their research fields and be able to identify their work. Furthermore,
when the general academic community in a particular discipline
is broken down into its specialist research fields, it would be
quite possible for all the researchers in a particular field to
know, or at least to be aware of one another's work. For instance,
research on governance and strategic planning in the English regions
was being conducted in the early years of the last decade by only
about a dozen researchers or research teams, two of which were
nationally and internationally eminent. Thus researchers in this
specialist field could probably identify one another's work when
asked to referee it anonymously. Hence professional rivalries
or personal animosities might distort a referee's judgment This
danger has been made more serious by the attempts by the Thatcher
Government and its successors to inject commercial competition
into academic life.
4. Secondly, referees have been known simply
to dismiss a colleague's work as rubbish without explaining themselves
or giving any reasons for dismissive judgments. Not only does
this mean that the author, whoever he or she may be, can derive
no constructive benefit from their comments; it is also extremely
distressing for them. It might cause a young researcher to give
up academic work or even to resort to self-harm. Such brief dismissals
of someone's work constitute a dereliction of the referee's scholarly
duty to assist colleagues in developing their research.
5. There is a possible danger that the refereeing
system may be used to enforce academic orthodoxies and deny dissident
or challenging researchers their right to publish. This may be
related to the problem of researchers being denied publication
because their findings are unwelcome to their research sponsors
or their Head of Department (CAFAS, passim). The Universities
and Colleges Union has to deal with such cases form time
to time.. In this context, referees who criticise an author
for not properly reviewing "the literature" should be
regarded with some suspicion. An editor has said that for his
journal, refereeing is intended mainly to ensure the "validity
and comprehensibility" of submitted articles: perhaps such
a limiting ordinance should be made more general. One effect of
refereeing has been that public academic controversies of old,
such as that between AJP Taylor and High Trevor Roper over the
origins of the Second World War, have largely vanished, which
may (or may not) be a cause for regret.
6. A major fault with the refereeing system is
the long delays that seem to be endemic in it, a point also made
by the Royal Society, whose committee stated that refereeing "has
been subject to criticism from outside the research community
for delaying or even preventing the disclosure of research results
that may have a bearing on the public interest" (2006: 12).
Senior academics, who bear the brunt of refereeing, are busy people
but they ought to be able and willing to turn requests for refereeing
journal articles around within around two weeks of receiving them.
One hopes that these delays never have an ulterior motive but
they are too common for delay not to be regarded as a significant
weakness in the system.
7. Referees
can make mistakes and of course, their judgments must inevitably
be subjective. One referee may be severely critical of a manuscript,
while another might recommend its acceptance for publication cum
laude.
8. Where a journal is seeking contributions from
practitioners as well as academics, as is the case, for example,
with the JUC/PAC journal, Public Policy & Administration,
submitting their contributions to referees may not be acceptable
to senior civil servants or local government officers who are
asked or encouraged to prepare contributions. This may also be
the case with Public Money & Management, which has
a large practitioner audience and seeks contributions from practitioners
of public administration and management, as well as from academics
in the field. Also, the delays involved in refereeing may mean
that such contributions will lose their topicality as a result
of the inevitable delay in publishing their work. In such cases,
a flexible editorial policy may be needed but not at the risk
of endangering the journal's credibility.
9. Possible components of a solution might include:
— Permitting
or requiring referees to sign their reports, which would also
enable referees, if they saw fit, to pass the manuscript and their
comments on them back to authors in the hope that they might be
helpful. The Committee and learned societies might go a bit further
than the Royal Society originally proposed. They suggested a voluntary
system whereby referees would be able to sign their reports if
they wished (2006: 12), in the interest of transparency, although
some referees may not wish to do so and some might not be prepared
to act as referees if they are asked to sign their reports. It
would not, therefore, be practicable to make signed reports obligatory
but it could be strongly encouraged.
— Editors
of journals and research funders could follow the Economic and
Social Research Council's example and ask authors themselves to
nominate one or two possible referees. The editors or research
funders could then nominate additional or alternative referees
as they thought fit but at least authors would then have some
reassurance that they might have "a friend at court".
However, all referees must be instructed to deliver honest, frank
assessments of authors' work.
— Editors
must be much firmer about setting and enforcing deadlines for
the receipt of referees' reports. I see encouraging signs that
some editors are trying to do this. If a referee has not completed
his or her report within, say, a month, the editor should withdraw
the request for refereeing and ask someone else to undertake it.
In extreme cases, one referee's report plus the editor's own judgment
may have to be sufficient to determine a MS's fate.
REFERENCES
CAFAS, passim: Campaign for Academic Freedom
and Standards, CAFAS Update, published periodically, contains
numerous accounts of the suppression of research by Heads of Department,
other university authorities or research sponsors (usually Government
departments).
Political Studies Association, 2006: Members'
Handbook. 2006.
Royal Society, 2006: Science and the public interest:
communicating the results of new scientific research to the public.
Professor Howard Elcock
21 February 2011
|
