Select Committee on Science and Technology Appendices to the Minutes of Evidence


Memorandum submitted by Professor Richard Green, Department of Economics, University of Hull

  1.  As currently constituted, the Research Assessment Exercise forces universities to gamble for public money. The Committee may wish to consider alternative methods of distributing taxpayers' funds. I also comment on an example of the law of unintended consequences.

  2.  Each unit of assessment chooses which of its academic staff to submit for the RAE. Those submitted are known as "research active"; those not submitted are designated "research inactive", but the designation is frequently misleading. It would be more accurate to call them "research less successful", as many researchers with respectable publication records will have been left out of the exercise.

  3.  The gamble in the RAE comes from the "cliff-edge" nature of the funding and grading decisions. A unit graded at 5 currently receives 3.375 "units" of funding per researcher submitted—the amount depends upon the subject, but it is roughly £20 thousand per researcher in the Arts and Social Sciences. If the unit was able to get a 5* by leaving out a small proportion of its staff, it would receive 20 per cent more per head. In Economics, I would guess that both Oxford and Cambridge tried this gamble, submitting about 90 per cent of their staff. If it had come off, they would have got 10 per cent more money than if they had submitted everyone and got a 5. In practice, neither department left out enough people to score a 5*, and so both got 10 per cent less money than if they had submitted everyone. I expect most academics could guess at similar stories in their own subjects. Lower down the funding scale, the cliff edges get larger—submitting one person too many and moving from a 5 to a 4 means that you will lose almost a third of your research funding. The funding councils are likely to change the funding scale in the near future, but the basic flaw remains.

  4.  I have heard that the original reason for allowing universities to choose whom to submit was to allow small research units to remain visible within larger "mainly teaching" departments. This was a sensible aim, but the consequences have been severe. Universities have discovered the gamble at the heart of the system, and waste hundreds of hours of senior staff time in deciding on their submission strategies. Where the gamble goes wrong, the university loses a significant amount of income. The personal and professional consequences for researchers who are deemed unworthy of submission can be severe.

  5.  Most importantly, the results are starting to lack credibility. Many departments have learned that if they submit two-thirds of their staff, they can hope to get a "4"—all of the research is of national excellence, with some of international standing. An identical department might submit everyone and qualify for a "3a"—two-thirds of its staff of national excellence. The two results might seem equivalent, and (at present) bring the same amount of funding, but the "4" department will be wrongly identified as the stronger.

  6.  I wish to propose a system that gets rid of the "cliff edges" in funding, and gives universities no reason not to submit all of their academic staff. RAE panels would identify "national" and "international" researchers, as they do at present. They would publish the number (but not the names) in each category in each unit of assessment. The funding councils would then give, say, £10,000 per year for every "national" researcher (the amount varying by subject) and two and a half times the first figure for every "international" researcher. The amounts distributed would be remarkably close to the current funding scale. If we need to skew funding towards "top" departments, then the amount for "national" researchers could be reduced, and that for "international" raised.

  7.  The amounts allocated might be similar to the existing system, but the incentives would be different. There would be no need to gamble. Universities could no longer lose money by submitting "marginal" researchers. Most of the time spent in agonising over the submission decisions could be used for more productive activities. The numbers of researchers assessed as national or international could be used to assemble league tables, but people who do not meet those criteria would not be publicly identified in the way that "research inactive" staff currently are.

  8.  My example of the law of unintended consequences concerns a second flaw in the system. By the 1996 RAE, it had become noticed that some universities were poaching good researchers shortly before the census date, and getting funding for them, while the institutions that had developed them received nothing. This was felt to be inappropriate, and so the A* system was developed. People who move between UK universities in the year before the RAE census can submit two publications for assessment on behalf of each institution, and count fully when that institution's quality is assessed. The institution employing them at the census date receives all the funding.

  9.  It is ironic that a system intended to discourage poaching seems almost designed to make it worse. The A* system gives less successful researchers the opportunity to be poached. Almost anyone will look better if they are assessed on their best two publications rather than on their best four. A department that has just hired a number of new staff will therefore tend to be given a higher quality rating than if it had not recently recruited, and will receive funding for all of its new members.

  10.  If the RAE really wanted to be neutral to staff movements, then all researchers should be judged on the same basis—presumably four publications. Staff who have left an institution within the past year should be included, whatever their destination, but counted at half their previous FTE when the department's overall quality and funding are determined. People who have joined an institution within the last year should also be counted at half their actual FTE.

  11.  If a unit loses a researcher and replaces them with someone of the same quality, this system would be neutral—they get half the funding for the departed researcher, and half for the new one. If someone leaves unexpectedly and has not been replaced by the census date, the department only loses half of the research funding for that person, reducing the impact of chance events. Departments will be discouraged from trying to expand rapidly just before the census date, because the proportion of the extra costs funded by the RAE will be lower. I have seen press speculation that some of the universities that spent heavily in the run-up to the RAE are now facing financial troubles, and hope that a more rational system could help prevent such problems in the future.

2 January 2002

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2002
Prepared 24 April 2002