Memorandum submitted by the Higher Education
Funding Council for England (HEFCE)
1. In its letter of 30 November, the clerk
to the committee requested a memorandum on the implications for
science and technology funding of the Research Assessment Exercise
"how the RAE has operated to date, how it
will inform the allocation of funds to higher education institutions
in 2002, and how it might operate in future alongside any other
mechanism for allocating research funding. Background information
on the way the Higher Education funding stream operates in respect
of science and technology would be very useful."
2. This paper represents HEFCE's response
to the committee's request.
3. This evidence is divided into seven sections
(supported by four annexes). For ease of reference, these are
Strength of UK research
4. Discusses the exceptional results achieved
by UK universities in RAE2001 and the supporting evidence validating
5. Describes the dual support system under
which publicly funded research is supported jointly by the funding
councils and the research councils. The results of the RAE determine
the allocation of research funds from the funding councils and
therefore have an impact upon the operation of the dual support
Selectivity and the RAE
6. Describes how the RAE supports the HEFCE's
policy of selective fundingproviding the most research
funds to the strongest university departments.
The history of the RAE
7. Describes the evolution of the exercise
and its impact upon the research base.
How the RAE works
8. Explains the RAE process and discusses
a number of aspects of the exercise which have caused comment.
The use of the RAE in funding in 2002-03
9. Discusses some of the options for implementing
the results of the RAE in HEFCE research funding in 2002-03.
The use of the RAE in funding after 2002-03
10. Discusses future funding policy and
the future of the RAE.
11. The results of the RAE, supported by
other evidence, demonstrate that UK research continues to improve
and to do so relative to other industrialised countries, despite
increasingly fierce competition.
12. Funding council research grant (which
is distributed on the basis of the RAE) is declining relative
to other sources of research funding. This means that funding
available for infrastructure continues to fall relative to the
demands placed upon that infrastructure.
13. The RAE is a system based on peer review.
It does not privilege any particular kind of research or researcher,
except inasmuch as the assessment panels have the power to define
their criteria for assessment.
14. The HEFCE is committed to implementing
the results of RAE2001 in the academic year 2002-03. The increase
in the volume of highly rated research means that it will not
be possible to maintain funding levels at every point of the rating
scale. However, the HEFCE has committed to protecting the unit
of resource for top rated (5*) departments.
15. The HEFCE will consider the future its
funding for research in the light of the outcome of the forthcoming
Spending Review and it will also consider the form of any future
research assessment exercise.
16. Public funding of research in the UK
is extremely cost-effective. A study published in 2000 showed
the UK ranked first amongst the leading industrialised nations
in the number of papers and the number of citations produced for
each $m of state funding.
What is more, the rate of return on publicly funded research is
hard to estimate but is agreed to be very high.
17. Further substantial improvements are
reflected in the results of the 2001 Research Assessment Exercise.
In 1996 32 per cent of staff that were submitted to the RAE as
research active were in departments rated 5 and 5*. In 2001 the
figure is 55 per cent. This improvement has been validated by
the opinion of the 290 overseas experts who were asked to comment
upon RAE panels' approach to grading. Of the 290, all but nine
supported the judgements of the panels.
18. This achievement is further validated
by new research which shows that the UK's share of the most cited
1 per cent of research papers has increased from 11 per cent to
18 per cent over the assessment period. The average citation rate
of UK papers relative to the rest of the world has improved by
12 per cent over the period. UK researchers are now cited at a
rate 38 per cent higher than the global averageand this
at a time when the competition is getting harder with research
developing fast in the Far East, and the volume of English language
publications from non-English speaking countries also rising very
19. With the exception of monies paid under
specific contracts from Government departments
and other public bodies, all public funding for research work
done by HEIs comes from one of two funding streams:
a. As part of the block grant from the higher
education funding bodies in the UK (HEFCE, HEFCW, SHEFC, and DELNI),
based on past performance as measured by the RAE.
b. Project grants allocated to a particular
researcher by the OST Research Councils (BBSRC, EPSRC, ESRC, MRC,
NERC, and PPARC) in response to proposals for programmes to carry
out future work.'
20. This is known as the dual support system.
Most major industrialised countries operate some form of dual
in which government support for research projects is underpinned
by funding streams which allow recipients discretion as to their
21. The dual support of research has the
it ensures that the research infrastructure
and capability is in place on which funders of projects can rely
for the work they seek to commission or fund
it ensures a plurality of decisions
about where funds should be allocated
it provides a degree of stability
in funding, which permits institutions to award permanent employment
contracts with more confidence than would be possible if they
relied on short-term and unpredictable project funding
A it enables institutions to exercise judgements
about where to put their research effort and, for example, to
revitalise departments that are in decline, or initiate new interdisciplinary
initiatives which may not attract support from Research Councils
it provides the only way of funding
the unconventional or unfashionable field, or the unknown researcher
who does not yet have the reputation to compete successfully for
Research Council grants.
22. In recent years the growth in project
funding (including funding from sources outside government) has
outstripped growth in funds allocated by the HE funding bodies
on the basis of the RAE. If this trend is allowed to continue
it will make it increasingly difficult for HEIs to support the
research capability needed to undertake high quality research
projects, as well negating the benefits of dual support. Under
these circumstances it is unsurprising that the implications of
the 2001 RAE results upon allocations of research funding have
been the cause of considerable anxiety within higher education.
23. A fuller discussion of dual support
can be found in Annex A.
24. The RAE supports the HEFCE's policy
of selectivity. This can be defined simply as ensuring that scarce
resources are directed towards those with the capacity to produce
research of the highest quality. By rating the quality of research
undertaken by university departments,
the RAE makes it possible for funding councils to discriminate
in their funding (this is done in a transparent way by formula).
25. Under the present arrangements, all
HEIs may compete for research funding, which is allocated selectively
on the basis of the quality of research. This means that virtually
all institutions receive some funds for research, but those with
research strengths in a large number of subjects receive the lion's
share, with 75 per cent of funds going to only 25 institutions.
At present, the principle which underpins research funding is
that of selectivity based on quality, wherever it is located,
not explicit concentration in a selected number of institutions.
26. The 1980s saw an explicit acknowledgement
of the role of science in a national strategy for economic competitiveness,
against a background of declining resources for HE. In response,
the University Grants Committee (UGC) developed an explicit and
formalised assessment processthe Research Assessment Exercise
(RAE)to replace the less transparent policy of selectivity
it exercised through its subject-based committees. The mechanism
was developed with two objectives in mind:
to identify what research was being
to identify what outputs were being
generated by this funding, that is, what research was being published.
27. The first RAE was undertaken in 1986.
In comparison to subsequent exercises it was a small scale affair.
What it did, in effect, was to standardise the information received
by the existing subject committees. The assessment process itself
did not become markedly more transparent or comprehensive until
28. Further exercises were held in 1989
and 1992, each a substantially larger undertaking than its predecessor.
The fourth exercise in 1996 considered the work of over 50,000
staff designated by HEIs as research active. These staff were
submitted to one of 69 Units of Assessment each with its own assessment
panel, and research outputs were rated on a scale of 1 to 5* (five
star). The 1996 exercise determined the allocation of over £4bn
over five years and its costs (including opportunity costs) have
been variously estimated at between £27m and £37m.
29. It is clear that by 1996 the RAE had
evolved from a quality assurance process to a competition for
funding. What is striking, however, is how successfully it retained
its original function of driving up standards through reputational
incentives, whilst at the same time enabling funds to be concentrated
in those departments best able to produce research of the highest
30. By the time of the most recent RAE in
2001, the exercise had become the principal means by which institutions
assured themselves of the quality of their research. It had also
evolved into an intense competition in which HEIs strive not only
for funding but also for prestige. RAE ratings are widely used
by institutions for marketing purposes, and are sometimes used
by business as a means of sifting potential collaborators and
contractors. The HEFCE believes that they also have an intrinsic
value to institutions and that the achievement of a strong RAE
rating can have a profound effect upon a department's self-confidence
31. In recent years the effects of these
financial and reputational incentives have become very apparent.
Work undertaken by the Higher Education Policy Unit (HEPU) at
the University of Leeds shows the extent to which the funding
from other sources correlates with RAE ratings. More recently,
the effects of the RAE upon research quality have become apparent
as noted above (paragraphs 16-18).
32. It would be fair to say that the improvement
in the RAE ratings achieved by HEIs in the 2001 RAE was not wholly
unexpected. In 2000, HEPU concluded that the RAE had transformed
the management of research within institutions in the following
a. There have been major changes in attitude
and strategy, underpinning significant moves towards more conscious
and active management of the research environment.
b. Management of the research environment
has improved (though research itself is still comparatively `unmanaged')
in both the social sciences and the natural sciences.
c. Structures have been developed that ensure
better accountability and can support a more strategic approach
to the development of research policy (for example establishing
research committees at both institutional and departmental or
d. Large institutions have increased their
share of research students and dedicated research workers more
than they have increased their share of funds, suggesting that
research management has promoted improved efficiency.
e. Institutions with a generally high quality
research base are focusing management attention on, and making
financial investments in, 4-rated departments in an attempt to
raise them to 5/5* status.
f. Institutions with a number of poorly-rated
departments often seek to increase quality by merging less successful
departments with those that were highly-rated, in the expectation
that exposure to the new culture and management will raise the
overall level of the merged entitythereby creating viable
and dynamic units capable of securing appropriate on-going levels
of research funding. This has had the added benefit of creating
broadly-based interdisciplinary and multidisciplinary units.
33. In the light of the very strong results
of the 2001 RAE, the HEFCE approached a sample of the most research-intensive
institutions for details of their research strategy in the years
leading up to the census point (March 2001). It is clear that
the need to compete effectively in the RAE means that institutions
have actively managed their research by setting specific targets,
developing benchmarking systems, and focusing on staff managementfor
example most had programmes of early retirement, and replaced
significant numbers of their less research active staff with new
blood. This has enabled an improvement in research capacity in
a very real sense.
34. The RAE is essentially a peer review
processor rather a collection of peer review processes.
Research in the UK is divided into 68 subject areas or Units of
Assessment (UoAs). An assessment panel is recruited to examine
research in each of these areas.
35. Higher education institutions are invited
to make submissions to as many UoAs as they choose. Submissions
are made in a standard format. Institutions are asked to provide
the names of staff (`research active staff') they wish to enter
in each UoA, and up to four research outputs
for each person. In making their assessments, panels are allowed
to consider only those research outputs mentioned in the submission
and the other data requested by the exercise, which includes information
about research income, research student numbers and an account
of the institution's research strategy.
36. There is no upper or lower limit on
the number of UoAs an institution can submit to. Nor is there
any limit on the number of staff submitted as research active
although data are published on the proportion of staff submitted
as research active.
37. Panels score each submission on a 7
point scale. The lowest grade is 1 and the highest 5* (five star).
Each panel is asked to produce and publish a set of assessment
criteria and working methods, to which it is bound to adhere.
38. The 2001 RAE was strengthened in a number
of respects to address some of the concerns which had been previously
expressed about aspects of the exercise.
39. It is sometimes suggested that the RAE
assessments reward those who have published most prolifically.
In fact, submissions can list a maximum of only four publications
per staff member. Panels are not allowed to consider any publications
which are not listed and have to justify their decisions as being
consistent with the content of the submission.
40. It is therefore unsurprising that the
editors of major international journals report that the RAE has
not had a significant distorting effect upon publication behaviour.
41. The RAE has been continually modified
to ensure that research which straddles two or more Units of Assessment
is not penalised by the disciplinary structure of the assessment.
Modifications for 2001 include a facility for panels to seek advice
from other panels or external advisers when considering outputs.
42. Bibliometric analyses shows that the
number of research papers authored by members of more than one
discipline is higher in the UK than in most other countries, and
that top-rated submissions are no less likely to contain a high
number of such papers than any other submissions.
Consistency of scores
43. While it can never be confidently said
that scores awarded by panels in different subjects, applying
different criteria appropriate to their subjects, are exactly
the same in terms of the standards they apply, we introduced a
number of measures to give greater confidence about this. In particular
we established umbrella groups of cognate panel chairs to ensure
that there were common approaches in similar subjects; and we
required all panels to appoint corresponding members overseas,
to whom they were obliged to refer any submissions to which they
proposed to award the highest scores.
44. It has been suggested that the RAE indirectly
promotes disadvantage for women and academics from ethnic minorities.
45. The RAE is perhaps the most transparent
peer review process anywhere in the world and we believe that
this transparency offers the best protection against inadvertent
discrimination. Furthermore, for the 2001 RAE panels were encouraged
to consider particular circumstances which may account for a researcher
not being able to include four substantial publications in the
RAE. This will offer protection to researchers who have taken
career breaks during the assessment period (who are, disproportionately,
46. Nevertheless, the HEFCE recognises that
the HE sector has not been as successful as it would like to have
been in increasing the numbers of women and members of minority
groups in senior positions. It has made the release of the additional
funds for staff development conditional upon the submission of
plans which adequately address equal opportunities issues, and
has published these plans and good practice guidance derived from
them. Furthermore, it is poised to announce a major research programme
designed to uncover the reasons for under-representation. Whilst
there is not yet any credible evidence that the RAE is part of
the problem, this will be considered as part of the research.
47. Concern has been expressed that the
RAE has encouraged an academic transfer market, with institutions
trying to increase their chances of high ratings in the short
term by recruiting staff with an outstanding research record shortly
before an RAE.
48. We do not believe that the movement
in the UK is a major problem. Indeed, the level of movement overall
is rather low, among all grades of staff in all types of institution,
and possibly so low as to be inconsistent with maintaining dynamism
in the sector. It is also lower than that which occurs, for example,
in the US or in some industrial research laboratories.
49. However, the funding bodies are aware
of the difficulties which have in the past been caused when key
members of staff move on in the period immediately preceding an
RAE. For this reason it was decided that staff who moved in the
year prior to the 2001 RAE may be cited by both the exporting
and importing institutions. 2,709 of these `A*' staff were returned
to the 2001 exercise. They will be counted in the funding calculation
for both institutions.
RAE IN FUNDING
50. The HEFCE funding model distributes
funds between subjects on the basis of fundable volume and cost
and within subjects on the basis of volume and quality.
51. Definitions of these terms and a more
detailed account of the funding model is given in Annex B.
52. The results of the 2001 RAE will drive
funding allocations from 2002-03. This will necessitate some hard
decisions, given that HEIs have invested heavily in order to produce
some extremely good results, whereas there is, at present, no
additional funding available to reward improved performance.
53. A decision on the fine detail of the
funding formula has yet to be taken. However, having considered
the results, the HEFCE Board has stated its funding principles
for 2002-03 as follows:
the average funding per unit of volume
for 5* departments will not be allowed to fall
any major funding changes at institutional
level will be moderated by a system of caps and safety nets taking
teaching and research income together
the average unit of funding for the
4 and 5 grades will fall. The reduction for the 5 grade will be
less severe than that for the 4 grade.
any reductions in the unit of funding
for the 4 and 5 grades will be revisited when the results of the
Government's Spending Review are known
some funding will be given to departments
rated 3b and 3a on a basis yet to be determined.
54. The HEFCE's most recent statement on
funding in 2002-03 is reproduced as Annex C.
RAE IN FUNDING
55. The elements of the funding model will
be reviewed over the coming year.
56. The weightings attached to each quality
band for the 2002-03 funding model will be decided at the HEFCE
Board's meetings in January and February. The weights to be used
from 2003-04 will be decided in the light of the 2002 Spending
Review, taking due account of the Government's cross-cutting review
of science policy.
57. It is likely that for 2002-03 the elements
which make up the volume measure will remain unchanged. They will
be reviewed over the coming year to ensure they are still fit
for purpose. It has been argued that the inclusion of research
students provides excessive encouragement for HEIs to recruit
such students. We will be looking in detail at this question,
in particular, over the coming months. We will also be looking
at the support provided for work undertaken by universities for
58. The assignation of subjects to cost
bands and the weightings attached to each band are due for review.
This is necessary as the relative costs of research in different
fields change from time to time. Current cost weights are based
upon an assessment undertaken in 2000-01. This review of cost
weights will be complete by March 2002. Details of the current
cost bands are given in Annex D.
The future of the RAE
59. By any measure, the RAE has been extremely
successful. It has helped to drive up research quality. It has
also gained the acceptance of the research community and its stakeholders.
In 2000 the HEFCE asked all interested parties to respond to a
consultation on its research policy. Faced with the proposition
that `there should continue to be a research assessment process
based on peer review, building on the foundations of the RAE',
98 per cent of respondents agreed.
60. This suggests that change should be
evolutionary rather than radical. However, the HEFCE is moving
towards the view that change may be needed for the following reasons:
a. Given that 55 per cent of research active
staff are now in departments rated 5 and 5*, it is arguable that
the RAE no longer provides enough discrimination to enable funding
to be allocated sufficiently selectively.
b. The RAE has helped to build capacity,
particularly in institutions without a research tradition. Whilst
there are still areas where capacity building remains a priority,
the overwhelming need now is to ensure that those capable of competing
with the best in the world are resourced to do so. There is some
evidence that the capital costs of undertaking world-class research
have been rising rapidly,
and the HEFCE believes that wage costs in many disciplines share
a similar tendency. This suggests that we should enable those
in receipt of funding for research to remain globally competitive.
The type of research assessment which would support such a policy
may be rather different from the RAE, which was designed for a
c. Any assessment process, particularly one
as important to its subjects as the RAE, will distort the very
thing it intends to measure. The RAE has lasted remarkably well,
but after five exercises it is timely for the funding bodies to
review whether it is likely to continue to be as effective an
instrument in future as it has been in the past.
d. Data from the Transparency Review of higher
education demonstrate that HEIs are losing £450m a year on
publicly funded research. Under these circumstances, it is necessary
to consider the ways in which HEFCE funding interacts with research
project funding (this issue is more fully discussed in Annex A).
The HEFCE is also reconsidering the funding it provides for the
recruitment of PhD students, and the means of assuring that it
funds only those institutions providing training of a suitable
standard. Any changes in policy resulting from these review processes
may alter the nature of the funding method and therefore the nature
of the assessment needed to support it.
61. For these reasons the HEFCE will be
recommending to its partners in the other UK funding bodies that
there is a fundamental reassessment of the form of research assessment
which can best deliver their individual and shared objectives.
17th January 2002
1 Katz 2000 (cited in FRR). Back
SPRU (2000) `Research contribution to the economy' (available
for download on the HEFCE web-site, www.hefce.ac.uk under `Research'
then `Fundamental review') estimates a rate of return of 30 per
cent. Other estimates have given similar figures. The HEFCE would
accept that the science of measuring the rate of return on research
is in its infancy. Nevertheless, the figures are striking. Back
The Department of Health in particular offers some streams of
funding which are not tied to specific projects. Back
Coopers & Lybrand report on indirect costs of OST Research
Council projects and programmes, March 1998. Back
SPRU (2000) `International approaches to research policy and
The term `department' may be slightly misleading. In the absence
of a more accurate term, we have described the research active
staff submitted to the RAE by an HEI under any one Unit of Assessment
as a `department'. In practice, however, HEIs are free to divide
the staff employed in any one department between several UoAs,
or to submit staff employed in more than one department to a single
In relative terms, the RAE is not an expensive exercise. The
most generous estimate of the costs of the exercise places them
at 0.8 per cent of the total funds distributed on the basis of
the exercise. This compares extremely favourably with the transaction
costs associated with bidding or tendering processes. Back
Research outputs are usually, but not always, publications. Other
types of output-including films, exhibitions or contributions
to television programmes-are increasingly common. Back
HEFCE 00/37 `Review of research' annex J, table J5. Back
HEFCE 00/37 `Review of research' annex H tables H4 and H5. Back
PREST (2000) `Impact of the Research Assessment Exercise' found
that not only were levels of movement relatively low but that
non-research active staff are more likely to move than research
active staff. PREST did, however, find evidence that the RAE has
a small effect on the timing of staff movements. Back
PREST 2000. Back