Select Committee on Science and Technology Minutes of Evidence

Memorandum submitted by the Higher Education Funding Council for England (HEFCE)


  1.  In its letter of 30 November, the clerk to the committee requested a memorandum on the implications for science and technology funding of the Research Assessment Exercise explaining

    "how the RAE has operated to date, how it will inform the allocation of funds to higher education institutions in 2002, and how it might operate in future alongside any other mechanism for allocating research funding. Background information on the way the Higher Education funding stream operates in respect of science and technology would be very useful."

  2.  This paper represents HEFCE's response to the committee's request.


  3.  This evidence is divided into seven sections (supported by four annexes). For ease of reference, these are described below.

Strength of UK research

  4.  Discusses the exceptional results achieved by UK universities in RAE2001 and the supporting evidence validating the results.

Dual support

  5.  Describes the dual support system under which publicly funded research is supported jointly by the funding councils and the research councils. The results of the RAE determine the allocation of research funds from the funding councils and therefore have an impact upon the operation of the dual support system.

Selectivity and the RAE

  6.  Describes how the RAE supports the HEFCE's policy of selective funding—providing the most research funds to the strongest university departments.

The history of the RAE

  7.  Describes the evolution of the exercise and its impact upon the research base.

How the RAE works

  8.  Explains the RAE process and discusses a number of aspects of the exercise which have caused comment.

The use of the RAE in funding in 2002-03

  9.  Discusses some of the options for implementing the results of the RAE in HEFCE research funding in 2002-03.

The use of the RAE in funding after 2002-03

  10.  Discusses future funding policy and the future of the RAE.


  11.  The results of the RAE, supported by other evidence, demonstrate that UK research continues to improve and to do so relative to other industrialised countries, despite increasingly fierce competition.

  12.  Funding council research grant (which is distributed on the basis of the RAE) is declining relative to other sources of research funding. This means that funding available for infrastructure continues to fall relative to the demands placed upon that infrastructure.

  13.  The RAE is a system based on peer review. It does not privilege any particular kind of research or researcher, except inasmuch as the assessment panels have the power to define their criteria for assessment.

  14.  The HEFCE is committed to implementing the results of RAE2001 in the academic year 2002-03. The increase in the volume of highly rated research means that it will not be possible to maintain funding levels at every point of the rating scale. However, the HEFCE has committed to protecting the unit of resource for top rated (5*) departments.

  15.  The HEFCE will consider the future its funding for research in the light of the outcome of the forthcoming Spending Review and it will also consider the form of any future research assessment exercise.


  16.  Public funding of research in the UK is extremely cost-effective. A study published in 2000 showed the UK ranked first amongst the leading industrialised nations in the number of papers and the number of citations produced for each $m of state funding[1]. What is more, the rate of return on publicly funded research is hard to estimate but is agreed to be very high.[2]

  17.  Further substantial improvements are reflected in the results of the 2001 Research Assessment Exercise. In 1996 32 per cent of staff that were submitted to the RAE as research active were in departments rated 5 and 5*. In 2001 the figure is 55 per cent. This improvement has been validated by the opinion of the 290 overseas experts who were asked to comment upon RAE panels' approach to grading. Of the 290, all but nine supported the judgements of the panels.

  18.  This achievement is further validated by new research which shows that the UK's share of the most cited 1 per cent of research papers has increased from 11 per cent to 18 per cent over the assessment period. The average citation rate of UK papers relative to the rest of the world has improved by 12 per cent over the period. UK researchers are now cited at a rate 38 per cent higher than the global average—and this at a time when the competition is getting harder with research developing fast in the Far East, and the volume of English language publications from non-English speaking countries also rising very rapidly.


  19.  With the exception of monies paid under specific contracts from Government departments[3] and other public bodies, all public funding for research work done by HEIs comes from one of two funding streams:

    a.  As part of the block grant from the higher education funding bodies in the UK (HEFCE, HEFCW, SHEFC, and DELNI), based on past performance as measured by the RAE.

    b.  Project grants allocated to a particular researcher by the OST Research Councils (BBSRC, EPSRC, ESRC, MRC, NERC, and PPARC) in response to proposals for programmes to carry out future work.'[4]

  20.  This is known as the dual support system. Most major industrialised countries operate some form of dual support system[5] in which government support for research projects is underpinned by funding streams which allow recipients discretion as to their use.

  21.  The dual support of research has the following benefits:

    —  it ensures that the research infrastructure and capability is in place on which funders of projects can rely for the work they seek to commission or fund

    —  it ensures a plurality of decisions about where funds should be allocated

    —  it provides a degree of stability in funding, which permits institutions to award permanent employment contracts with more confidence than would be possible if they relied on short-term and unpredictable project funding

A it enables institutions to exercise judgements about where to put their research effort and, for example, to revitalise departments that are in decline, or initiate new interdisciplinary initiatives which may not attract support from Research Councils

    —  it provides the only way of funding the unconventional or unfashionable field, or the unknown researcher who does not yet have the reputation to compete successfully for Research Council grants.

  22.  In recent years the growth in project funding (including funding from sources outside government) has outstripped growth in funds allocated by the HE funding bodies on the basis of the RAE. If this trend is allowed to continue it will make it increasingly difficult for HEIs to support the research capability needed to undertake high quality research projects, as well negating the benefits of dual support. Under these circumstances it is unsurprising that the implications of the 2001 RAE results upon allocations of research funding have been the cause of considerable anxiety within higher education.

  23.  A fuller discussion of dual support can be found in Annex A.


  24.  The RAE supports the HEFCE's policy of selectivity. This can be defined simply as ensuring that scarce resources are directed towards those with the capacity to produce research of the highest quality. By rating the quality of research undertaken by university departments[6], the RAE makes it possible for funding councils to discriminate in their funding (this is done in a transparent way by formula).

  25.  Under the present arrangements, all HEIs may compete for research funding, which is allocated selectively on the basis of the quality of research. This means that virtually all institutions receive some funds for research, but those with research strengths in a large number of subjects receive the lion's share, with 75 per cent of funds going to only 25 institutions. At present, the principle which underpins research funding is that of selectivity based on quality, wherever it is located, not explicit concentration in a selected number of institutions.


  26.  The 1980s saw an explicit acknowledgement of the role of science in a national strategy for economic competitiveness, against a background of declining resources for HE. In response, the University Grants Committee (UGC) developed an explicit and formalised assessment process—the Research Assessment Exercise (RAE)—to replace the less transparent policy of selectivity it exercised through its subject-based committees. The mechanism was developed with two objectives in mind:

    —  to identify what research was being funded

    —  to identify what outputs were being generated by this funding, that is, what research was being published.

  27.  The first RAE was undertaken in 1986. In comparison to subsequent exercises it was a small scale affair. What it did, in effect, was to standardise the information received by the existing subject committees. The assessment process itself did not become markedly more transparent or comprehensive until 1989.

  28.  Further exercises were held in 1989 and 1992, each a substantially larger undertaking than its predecessor. The fourth exercise in 1996 considered the work of over 50,000 staff designated by HEIs as research active. These staff were submitted to one of 69 Units of Assessment each with its own assessment panel, and research outputs were rated on a scale of 1 to 5* (five star). The 1996 exercise determined the allocation of over £4bn over five years and its costs (including opportunity costs) have been variously estimated at between £27m and £37m.[7]

  29.  It is clear that by 1996 the RAE had evolved from a quality assurance process to a competition for funding. What is striking, however, is how successfully it retained its original function of driving up standards through reputational incentives, whilst at the same time enabling funds to be concentrated in those departments best able to produce research of the highest quality.

  30.  By the time of the most recent RAE in 2001, the exercise had become the principal means by which institutions assured themselves of the quality of their research. It had also evolved into an intense competition in which HEIs strive not only for funding but also for prestige. RAE ratings are widely used by institutions for marketing purposes, and are sometimes used by business as a means of sifting potential collaborators and contractors. The HEFCE believes that they also have an intrinsic value to institutions and that the achievement of a strong RAE rating can have a profound effect upon a department's self-confidence and reputation.

  31.  In recent years the effects of these financial and reputational incentives have become very apparent. Work undertaken by the Higher Education Policy Unit (HEPU) at the University of Leeds shows the extent to which the funding from other sources correlates with RAE ratings. More recently, the effects of the RAE upon research quality have become apparent as noted above (paragraphs 16-18).

  32.  It would be fair to say that the improvement in the RAE ratings achieved by HEIs in the 2001 RAE was not wholly unexpected. In 2000, HEPU concluded that the RAE had transformed the management of research within institutions in the following ways:

    a.  There have been major changes in attitude and strategy, underpinning significant moves towards more conscious and active management of the research environment.

    b.  Management of the research environment has improved (though research itself is still comparatively `unmanaged') in both the social sciences and the natural sciences.

    c.  Structures have been developed that ensure better accountability and can support a more strategic approach to the development of research policy (for example establishing research committees at both institutional and departmental or school level).

    d.  Large institutions have increased their share of research students and dedicated research workers more than they have increased their share of funds, suggesting that research management has promoted improved efficiency.

    e.  Institutions with a generally high quality research base are focusing management attention on, and making financial investments in, 4-rated departments in an attempt to raise them to 5/5* status.

    f.  Institutions with a number of poorly-rated departments often seek to increase quality by merging less successful departments with those that were highly-rated, in the expectation that exposure to the new culture and management will raise the overall level of the merged entity—thereby creating viable and dynamic units capable of securing appropriate on-going levels of research funding. This has had the added benefit of creating broadly-based interdisciplinary and multidisciplinary units.

  33.  In the light of the very strong results of the 2001 RAE, the HEFCE approached a sample of the most research-intensive institutions for details of their research strategy in the years leading up to the census point (March 2001). It is clear that the need to compete effectively in the RAE means that institutions have actively managed their research by setting specific targets, developing benchmarking systems, and focusing on staff management—for example most had programmes of early retirement, and replaced significant numbers of their less research active staff with new blood. This has enabled an improvement in research capacity in a very real sense.


  34.  The RAE is essentially a peer review process—or rather a collection of peer review processes. Research in the UK is divided into 68 subject areas or Units of Assessment (UoAs). An assessment panel is recruited to examine research in each of these areas.

  35.  Higher education institutions are invited to make submissions to as many UoAs as they choose. Submissions are made in a standard format. Institutions are asked to provide the names of staff (`research active staff') they wish to enter in each UoA, and up to four research outputs[8] for each person. In making their assessments, panels are allowed to consider only those research outputs mentioned in the submission and the other data requested by the exercise, which includes information about research income, research student numbers and an account of the institution's research strategy.

  36.  There is no upper or lower limit on the number of UoAs an institution can submit to. Nor is there any limit on the number of staff submitted as research active although data are published on the proportion of staff submitted as research active.

  37.  Panels score each submission on a 7 point scale. The lowest grade is 1 and the highest 5* (five star). Each panel is asked to produce and publish a set of assessment criteria and working methods, to which it is bound to adhere.

  38.  The 2001 RAE was strengthened in a number of respects to address some of the concerns which had been previously expressed about aspects of the exercise.

Publication behaviour

  39.  It is sometimes suggested that the RAE assessments reward those who have published most prolifically. In fact, submissions can list a maximum of only four publications per staff member. Panels are not allowed to consider any publications which are not listed and have to justify their decisions as being consistent with the content of the submission.

  40.  It is therefore unsurprising that the editors of major international journals report that the RAE has not had a significant distorting effect upon publication behaviour.[9]

Interdisciplinary research

  41.  The RAE has been continually modified to ensure that research which straddles two or more Units of Assessment is not penalised by the disciplinary structure of the assessment. Modifications for 2001 include a facility for panels to seek advice from other panels or external advisers when considering outputs.

  42.  Bibliometric analyses shows that the number of research papers authored by members of more than one discipline is higher in the UK than in most other countries, and that top-rated submissions are no less likely to contain a high number of such papers than any other submissions.[10]

Consistency of scores

  43.  While it can never be confidently said that scores awarded by panels in different subjects, applying different criteria appropriate to their subjects, are exactly the same in terms of the standards they apply, we introduced a number of measures to give greater confidence about this. In particular we established umbrella groups of cognate panel chairs to ensure that there were common approaches in similar subjects; and we required all panels to appoint corresponding members overseas, to whom they were obliged to refer any submissions to which they proposed to award the highest scores.

Equal opportunities

  44.  It has been suggested that the RAE indirectly promotes disadvantage for women and academics from ethnic minorities.

  45.  The RAE is perhaps the most transparent peer review process anywhere in the world and we believe that this transparency offers the best protection against inadvertent discrimination. Furthermore, for the 2001 RAE panels were encouraged to consider particular circumstances which may account for a researcher not being able to include four substantial publications in the RAE. This will offer protection to researchers who have taken career breaks during the assessment period (who are, disproportionately, women).

  46.  Nevertheless, the HEFCE recognises that the HE sector has not been as successful as it would like to have been in increasing the numbers of women and members of minority groups in senior positions. It has made the release of the additional funds for staff development conditional upon the submission of plans which adequately address equal opportunities issues, and has published these plans and good practice guidance derived from them. Furthermore, it is poised to announce a major research programme designed to uncover the reasons for under-representation. Whilst there is not yet any credible evidence that the RAE is part of the problem, this will be considered as part of the research.

Staff movement

  47.  Concern has been expressed that the RAE has encouraged an academic transfer market, with institutions trying to increase their chances of high ratings in the short term by recruiting staff with an outstanding research record shortly before an RAE.

  48.  We do not believe that the movement in the UK is a major problem. Indeed, the level of movement overall is rather low, among all grades of staff in all types of institution, and possibly so low as to be inconsistent with maintaining dynamism in the sector. It is also lower than that which occurs, for example, in the US or in some industrial research laboratories.[11]

  49.  However, the funding bodies are aware of the difficulties which have in the past been caused when key members of staff move on in the period immediately preceding an RAE. For this reason it was decided that staff who moved in the year prior to the 2001 RAE may be cited by both the exporting and importing institutions. 2,709 of these `A*' staff were returned to the 2001 exercise. They will be counted in the funding calculation for both institutions.


  50.  The HEFCE funding model distributes funds between subjects on the basis of fundable volume and cost and within subjects on the basis of volume and quality.

  51.  Definitions of these terms and a more detailed account of the funding model is given in Annex B.

  52.  The results of the 2001 RAE will drive funding allocations from 2002-03. This will necessitate some hard decisions, given that HEIs have invested heavily in order to produce some extremely good results, whereas there is, at present, no additional funding available to reward improved performance.

  53.  A decision on the fine detail of the funding formula has yet to be taken. However, having considered the results, the HEFCE Board has stated its funding principles for 2002-03 as follows:

    —  the average funding per unit of volume for 5* departments will not be allowed to fall

    —  any major funding changes at institutional level will be moderated by a system of caps and safety nets taking teaching and research income together

    —  the average unit of funding for the 4 and 5 grades will fall. The reduction for the 5 grade will be less severe than that for the 4 grade.

    —  any reductions in the unit of funding for the 4 and 5 grades will be revisited when the results of the Government's Spending Review are known

    —  some funding will be given to departments rated 3b and 3a on a basis yet to be determined.

  54.  The HEFCE's most recent statement on funding in 2002-03 is reproduced as Annex C.


  55.  The elements of the funding model will be reviewed over the coming year.


  56.  The weightings attached to each quality band for the 2002-03 funding model will be decided at the HEFCE Board's meetings in January and February. The weights to be used from 2003-04 will be decided in the light of the 2002 Spending Review, taking due account of the Government's cross-cutting review of science policy.


  57.  It is likely that for 2002-03 the elements which make up the volume measure will remain unchanged. They will be reviewed over the coming year to ensure they are still fit for purpose. It has been argued that the inclusion of research students provides excessive encouragement for HEIs to recruit such students. We will be looking in detail at this question, in particular, over the coming months. We will also be looking at the support provided for work undertaken by universities for charities.


  58.  The assignation of subjects to cost bands and the weightings attached to each band are due for review. This is necessary as the relative costs of research in different fields change from time to time. Current cost weights are based upon an assessment undertaken in 2000-01. This review of cost weights will be complete by March 2002. Details of the current cost bands are given in Annex D.

The future of the RAE

  59.  By any measure, the RAE has been extremely successful. It has helped to drive up research quality. It has also gained the acceptance of the research community and its stakeholders. In 2000 the HEFCE asked all interested parties to respond to a consultation on its research policy. Faced with the proposition that `there should continue to be a research assessment process based on peer review, building on the foundations of the RAE', 98 per cent of respondents agreed.

  60.  This suggests that change should be evolutionary rather than radical. However, the HEFCE is moving towards the view that change may be needed for the following reasons:

    a.  Given that 55 per cent of research active staff are now in departments rated 5 and 5*, it is arguable that the RAE no longer provides enough discrimination to enable funding to be allocated sufficiently selectively.

    b.  The RAE has helped to build capacity, particularly in institutions without a research tradition. Whilst there are still areas where capacity building remains a priority, the overwhelming need now is to ensure that those capable of competing with the best in the world are resourced to do so. There is some evidence that the capital costs of undertaking world-class research have been rising rapidly,[12] and the HEFCE believes that wage costs in many disciplines share a similar tendency. This suggests that we should enable those in receipt of funding for research to remain globally competitive. The type of research assessment which would support such a policy may be rather different from the RAE, which was designed for a different environment.

    c.  Any assessment process, particularly one as important to its subjects as the RAE, will distort the very thing it intends to measure. The RAE has lasted remarkably well, but after five exercises it is timely for the funding bodies to review whether it is likely to continue to be as effective an instrument in future as it has been in the past.

    d.  Data from the Transparency Review of higher education demonstrate that HEIs are losing £450m a year on publicly funded research. Under these circumstances, it is necessary to consider the ways in which HEFCE funding interacts with research project funding (this issue is more fully discussed in Annex A). The HEFCE is also reconsidering the funding it provides for the recruitment of PhD students, and the means of assuring that it funds only those institutions providing training of a suitable standard. Any changes in policy resulting from these review processes may alter the nature of the funding method and therefore the nature of the assessment needed to support it.

  61.  For these reasons the HEFCE will be recommending to its partners in the other UK funding bodies that there is a fundamental reassessment of the form of research assessment which can best deliver their individual and shared objectives.

17th January 2002

1   Katz 2000 (cited in FRR). Back

2   SPRU (2000) `Research contribution to the economy' (available for download on the HEFCE web-site, under `Research' then `Fundamental review') estimates a rate of return of 30 per cent. Other estimates have given similar figures. The HEFCE would accept that the science of measuring the rate of return on research is in its infancy. Nevertheless, the figures are striking. Back

3   The Department of Health in particular offers some streams of funding which are not tied to specific projects. Back

4   Coopers & Lybrand report on indirect costs of OST Research Council projects and programmes, March 1998. Back

5   SPRU (2000) `International approaches to research policy and funding'. Back

6   The term `department' may be slightly misleading. In the absence of a more accurate term, we have described the research active staff submitted to the RAE by an HEI under any one Unit of Assessment as a `department'. In practice, however, HEIs are free to divide the staff employed in any one department between several UoAs, or to submit staff employed in more than one department to a single UoA. Back

7   In relative terms, the RAE is not an expensive exercise. The most generous estimate of the costs of the exercise places them at 0.8 per cent of the total funds distributed on the basis of the exercise. This compares extremely favourably with the transaction costs associated with bidding or tendering processes. Back

8   Research outputs are usually, but not always, publications. Other types of output-including films, exhibitions or contributions to television programmes-are increasingly common. Back

9   HEFCE 00/37 `Review of research' annex J, table J5. Back

10   HEFCE 00/37 `Review of research' annex H tables H4 and H5. Back

11   PREST (2000) `Impact of the Research Assessment Exercise' found that not only were levels of movement relatively low but that non-research active staff are more likely to move than research active staff. PREST did, however, find evidence that the RAE has a small effect on the timing of staff movements. Back

12   PREST 2000. Back

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2002
Prepared 7 March 2002