Select Committee on Science and Technology Minutes of Evidence

Memorandum submitted by the Association of University Teachers


  The Association of University Teachers (AUT) represents over 43,500 academic and academic-related staff in UK universities, colleges and research institutes. We welcome the chance to respond to the inquiry into the research assessment exercise (RAE). Higher education has become a powerful element within the UK research system and of central concern to the economy and national life. Nearly £1 billion of research funding is allocated each year using the results of the RAE, and the bulk of this money is allocated to science and technology. The status, role and funding of the RAE therefore has major implications for the UK science base.

  The 2001 RAE results show a marked increase in the proportion of departments whose research is judged to be of international excellence. The credit is due to our academic staff—supported by academic related colleagues—for the world-class standard of the research produced in UK higher education. While we welcome the government's emphasis on the importance of world-class research, we regret their decision not to fully fund the improvements in quantity and quality that have been revealed in the 2001 RAE results. We are also disappointed that the Higher Education Funding Council for England (HEFCE) have proposed a further concentration of research funding in a small number of 5* departments. This short-term palliative will create a longer-term crisis in UK science as new opportunities to conduct research become stifled. The science base is too important to the economic, cultural and social health of the UK to allow this ossification and stagnation to occur.

1.  What are the Problems with the RAE?

  The successful distribution of public funding for university research involves striking a balance between the need, on the one hand, to ensure accountability and 'value for money' in the use of public funds, and, on the other, to encourage creative enquiry and flexibility. We believe that the balance has gone too far in the direction of audit, assessment and instant evaluation over the last decade or so. The main expression of this process has been the concentration of funding council support through the mechanism of the RAE and the closely connected phenomenon of the weakening of the dual support system.

  We therefore wish to preface our comments by reiterating very strongly our continuing opposition to the RAE. As a trade union, we are particularly concerned about the immediate effects of the 2001 exercise on staff morale and members' jobs, but we also believe there are inherent problems with the RAE as a means of allocating research funding. Our members experience of research selectivity in the context of funding cuts has been overwhelmingly one of divisiveness, unfairness and demoralisation. As a result, we believe that fundamental changes are needed to the way research is currently funded and assessed.

1.1  Too much bureaucracy

  The UK RAE is the largest and most comprehensive evaluation programme of its type in the world. For example, in the 2001 exercise, sixty assessment panels assessed approximately 200,000 publications submitted by almost 50,000 academics. The UK research assessment process is also unusual in that it involves a direct link with the funding allocation mechanism. Only a small number of other countries, such as Australia, the Slovak Republic and Poland, carry out a research evaluation nationally and use the result to decide on the allocation of funding.1 The UK assessment process is also highly significant in terms of its contribution to overall research funding. For example, whereas nearly 20 per cent of UK higher education funding council funds in 2000-01 were allocated as a result of the RAE, less than 5 per cent were allocated via the Australian equivalent (the Institutional Grants Scheme).2 As a result, preparing for the RAE has become a colossal exercise in terms of staff time and resources.

  Unfortunately, official attempts to estimate the total cost of the 1996 exercise have come up with a highly conservative figure (£27.5 million). This estimation does not include the indirect costs incurred by the departments in organising and managing the submission. Other studies have highlighted the behavioural costs imposed by the higher education "audit culture". For example, a study by Ian McNay on the impact of the 1992 RAE revealed that 65 percent of staff surveyed thought that the pressures from the exercise had increased their stress levels.3

1.2  Too much selectivity

  A key objective of research policy is to strike a balance between on the one hand supporting research excellence, and, on the other, encouraging dynamism in the sector, allowing new subjects and centres to develop. We believe that the RAE has led to an over-concentration of research funding in a small number of leading departments and institutions (for example, 75 per cent of HEFCE research funds are allocated to just 26 higher education institutions). In the last two decades the degree of selectivity in UK university research funding has increased significantly. This contrasts with research funding in the US university system, where the trend has been in the opposite direction, and which is now less concentrated than its UK equivalent.4

  The RAE funds research on a retrospective basis only. As a result the exercise has a built-in element of irreversibility and self-reinforcement: a department that fares badly in one assessment loses funds and status and will find it extremely hard to re-establish itself before the next assessment, with the consequence that it then receives a further reduction in its research capacity and potential. The RAE also affects the ability of departments to compete for research support from outside sources, and so the trend towards concentration of funding is greatly strengthened.

  Underlying the old dual support system was a healthy scepticism about the ability of any funding system to "pick winners" in the research race. We now have a funding system which relies almost exclusively on past performance as a predictor of future research success. A retrospective element obviously has a part to play in picking winners but it not the only predictor and it does have peculiar limitations, such as its inherent tendency to favour the tried and tested lines of enquiry over the adventurous and innovative. Over-concentration of research money has deprived many talented researchers, especially relatively new entrants to the profession, of access to research support. The fact is that research potential is distributed very widely throughout our higher education institutions, but it is increasingly frustrated by a funding system that operates as if it were to be found only in 15 or so universities.

  Our belief that the degree of selectivity in university research funding was beginning to have a negative impact on the overall research capacity and potential of our universities was the main reason why we argued in 1998 for the postponement of the current RAE and the introduction of a "seedcorn fund" to stimulate research growth in some of the departments hardest hit by the selective effects of successive RAEs.5 It gives us little comfort to say that "we told you so", but if our advice had been heeded by the funding councils we would not now be facing the current, entirely predictable funding dilemma (see 1.3.2). Because of the current debacle, we recommend if not the abandonment of the RAE in its current form, then the postponement of the 2006 exercise.

1.3  The 2001 RAE

  On many measures UK higher education researchers are already among the best in the world. For example, the UK ranks first in the world in terms of the numbers of publications and citations generated per million dollars spent on research. The proportion of staff in 5 and 5* rated departments (that is, departments of international excellence in at least half of their research activity) in UK higher education institutions however has increased from 31 per cent in 1996 to 55 per cent in 2001.

1.3.1  Is the improvement genuine?

  We believe there has been a genuine increase in the international excellence of UK research. This improvement has been ratified by overseas assessors linked to the RAE panels (in all but three per cent of the cases, international experts confirmed the panels' judgement). Bibliometric analysis has also shown that the proportion of UK entries in the annual top 1 per cent of most highly cited papers in the world increased from 11 per cent in 1995 to 18 per cent in 2000.6 At the same time, we acknowledge that the improvement in RAE grades is also the result of more sophisticated institutional strategies. Compared to 1996, fewer researchers and departments were included in 2001 as institutions gambled on securing more money for fewer researchers rather than receiving less funding for a larger number of researchers. Institutional 'game playing' is an inevitable consequence of performance-based research funding, but it has had a detrimental impact on the increasing numbers of able staff who are excluded from the exercise (see section 1.4).

1.3.2  The impact of the funding shortfall on staff

  The increase in highly-rated research, without a corresponding increase in the level of funding, means that there will be a recurrent funding shortfall of £200 million for university research in the UK. At the time of writing (January 2002), HEFCE have indicated that they are only willing to maintain the unit of resource in 5* departments. The basis for funding research with a 3a, 3b, 4 or 5 rating is yet to be announced, but it seems inevitable that many departments will not receive the funding merited by their ratings. Many staff who have devoted their efforts and talents to improving the quantity and quality of their research now fear that their work will go unrewarded because of the need to fund the outcome of the 2001 RAE even more selectively than the normal translation of the research ratings into funding allocations would suggest.

  The allocation of funding proposed by the funding council will have a catastrophic impact on staff morale and motivation at a time when our institutions are already under intolerable pressures to dance to a variety of government tunes, not entirely harmonious. There is widespread cynicism among staff about the value of the various bureaucratic hoops that they are forced to jump through these days. In the case of the RAE, the university community have responded effectively to the government's challenge and delivered sizeable improvements in research quality. A failure to reward this success undermines the whole basis of the performance-based approach to funding. We cannot see how the RAE can possibly survive the current debacle, at least in anything resembling its present form.

  More seriously, from the point of view of our immediate concerns, is the potential impact of the RAE on our members' jobs. Some departments and institutions are likely to have funding shortfalls that they will find great difficulty in sustaining. It would be utterly unacceptable if the RAE were to lead to staff redundancies at a time when the government is demanding a further massive expansion in student numbers.

  Part of the problem lies with the government and their refusal to fund university research properly. Even with additional Science Research Investment Fund (SRIF) money, the continuing success of UK scientific research is threatened by lack of adequate funding and the poor condition of the basic research infrastructure and equipment in many universities. However, the RAE itself is also seriously flawed and we believe the funding councils have a responsibility to mitigate its impact on institutions and their staff. It is particularly important to ensure that the funding allocation protects institutions from sudden, significant falls in their grant.

1.4  Exclusion and division

  In addition to the funding shortfall, the association believes that there are inherent problems with the RAE. One of the most detrimental long-term effects of the exercise has been the creation of a "competitive, adversarial and punitive spirit" within the profession.7 The RAE promotes an invidious distinction between those whose work is submitted for assessment and those whose work is not. The 1996 exercise led to the victimisation of some staff who were said not to have contributed to the RAE and this happened even in some highly-rated departments.

  Universities were far more selective in the numbers of researchers (and departments) submitted to the 2001 assessment compared with previous exercises. We have many members who were told that they were unquestionably eligible to be entered in the 2001 exercise but would not be 'for strategic reasons'. These colleagues, often women at the early stages of their careers, are now officially deemed to be "non-research active". The exclusion of individuals from the recognition conferred by the status of "research active" has potentially harmful effects in terms of future promotion, job applications, grant applications, sabbaticals, etc.

  Other instances of "game playing" include transferring "research inactive" academic staff onto "other-related' grades" (in some cases with a teaching-only contract). This problem arises from the perceived need on the part of HEIs to decrease the total number of academic staff so that the proportion of staff submitted as research active is increased. This perceived need arises in the first place because RAE quality ratings are published alongside a banded score for the proportion of academic staff submitted as research active. In fact, the proportion data has only a marginal effect in terms of RAE ratings, and such tactics have been questioned by HEFCE. Moreover, this misjudgement is leading to actions which disadvantage some members of staff (for example, in relation to eligibility for early retirement).8

  Regrettably, in some departments, "non-research active" colleagues are treated as second class citizens, despite the fact that it may be precisely their contributions to teaching, pastoral care, curriculum development and administration, which enable their colleagues to concentrate on their research activities. The effect of this, apart from the unfairness and de-motivation expressed by individuals, is to undermine the professional collaboration and teamwork among academic and related staff on which the effective provision of high quality learning, teaching and research depends.

  A related issue concerns the status of contract research staff (CRS). Ninety-four per cent of "research only" staff in 1999/00 were on fixed-term contracts. HEFCE research shows that anatomy, physiology, pharmacology, biochemistry and veterinary science have particularly high proportions of staff on fixed-term contracts.9 RAE rules on counting CRS as 0.1 of a person unnecessarily distorts and devalues their role in the research productivity of an institution and perpetuates further their exclusion from the research culture of a department. The desire for full RAE returnees to achieve international standing and be seen to bring in research money often results in CRS being denied conference expenses, deliberately sidelined from being principal investigators on grant proposals and effectively pushed out of other means of research career development. The low recognition given to CRS in the RAE also encourages malpractice in designating authorship to published works that have been written in research teams. The RAE in its current form offers justification for the exploitation of the contractual vulnerability of CRS. AUT recommends that CRS are treated with parity to their permanent academic colleagues.

1.5  Impact on teaching and other academic activities

  One of the continuing concerns of the science and technology committee has been the quantity and quality of university science education. There are clearly specific issues that pertain to science, such as the poor quality of teaching labs and shortages in teaching and support staff. However, there is also a general problem that needs addressing: the relatively low status of teaching and learning in higher education. We have for many years argued that contributions to teaching should be valued equally to contributions to research, and that this should be reflected in all aspects of career progression, including promotion criteria. Establishing this change in culture has been an uphill struggle since the introduction of the RAE which has grossly distorted the higher education value system, not just in favour of research, but in favour of research output of particular kinds (ie those that count towards RAE ratings).

  In many institutions performance in the RAE has become the overwhelming focus of their activities because of the funding and prestige that it confers. The pressure on staff to contribute to the RAE has led to a tendency to neglect important activities which would normally be seen as part of a balanced university culture: development of courses and teaching materials, personal tutoring, reporting and monitoring student progress, training and mentoring teaching assistants, staff development, and so on.10 The dominance of the RAE has also hindered the development of contributions to the local, regional and national economies through applied research, training and consultancy.11

1.6  Gender difference and activity

  A key issue facing UK university science is the need to attract women into the profession and to enable them to progress onto the highest grades. One of the factors hindering gender equality in academia has been the disproportionate numbers of women who have been designated as "research inactive". Analysis of data from the higher education statistics agency, for example, suggests a significant under-representation of women in the 1996 RAE, including those in science departments.12 HEFCE's own evidence shows that the highest graded departments (5 & 5*) have a higher proportion of men than women. One of the recommendations of HEFCE's fundamental review of research, published in August 2000, was to consider the reasons for this under-representation and whether there are other groups who appear not to be realising their full research potential. We regret that this research has not yet been published, and now believe that more decisive action is required (see below).

RAE procedures on maternity leave

  One of our specific concerns is the treatment of women who have taken maternity leave. During preparations for the previous RAE in 1996 the association received complaints from a number of women members whose work was deemed "inadequate" for submission to the assessment. It was clear that in some of those cases the members' research output had been affected by periods spent away from research on maternity leave. For the 2001 exercise, the final RAE panel criteria and procedures did acknowledge the need to take individual staff circumstances into account, including maternity leave:

    "The situation of staff who have taken maternity leave or other career breaks, who hold part-time contracts, who are disabled or who have been absent for long periods through illness (where this is indicated by HEIs) will be taken into account in reaching overall judgements of quality where it is indicated in submissions." (para 2.18, p12, RAE 2001: Assessment Panels' Criteria and Working Methods, RAE 5/99)

  Although this was a helpful statement, only 20 per cent of the panels issued guidelines mentioning maternity leave as a possible factor in research performance. None of the science-based panels included a specific reference to maternity leave. In the run up the current exercise we received anecdotal evidence that heads of department were reluctant to implement these guidelines (on the grounds that panels were unlikely to adhere to them). Consequently, there remained a tendency to exclude maternity leavers, and returners from career breaks, in the 2001 exercise if they had not returned four items of research. We believe that the funding councils should issue stronger guidelines to protect the position of women who take maternity leave.

An equal opportunities audit

  In the 2001 RAE, HEFCE collected information on the gender of staff entered in the exercise, which would enable it to produce information on the performance of men and women in the RAE. Given that the names of selected staff are due to be published on the Internet in the spring of 2002, we see no reason why HEFCE is unable to publish this gender data as soon as possible. Publishing the raw data will enable analysis of whether women have suffered from discrimination in the RAE. In addition, we believe that the time is ripe for a comprehensive equal opportunities audit of the whole 2001 exercise. This audit should be undertaken by an independent body, such as the Equal Opportunities Commission, in conjunction with the newly-formed Equality Challenge Unit.


  To summarise our view: some selectivity in the distribution of research funds is necessary and given the limits on the availability of funds, inevitable, but the degree of selectivity and concentration of research funding practised over the last decade has now become wasteful of talent and conservative in its reliance on the "safe bet". We accept that there can be no return to the old dual support system in its original form which in essence distributed funding council money for research according to a headcount of student numbers in each institution. However, we believe that the case for extra funding to broaden our university research base is absolutely overwhelming. Our specific reforms for achieving this are as follows:

2.1  "Seedcorn" funding

  Investment in future and potential success is just as important a part of the process of funding research as is investment in existing excellence. We believe there should be a "seedcorn" fund available to university researchers for starting up new programmes. This should be aimed at counterbalancing to some extent the concentration effect of research selectivity. The present RAE ratings are a crude mechanism for evaluating departments. A low departmental rating can stifle research opportunities for all its members, even though some of them have great potential and exciting ideas. We need to be able to throw a lifebelt to these researchers before they are demotivated or leave the system in frustration. In the 1994-95 funding allocation, as an interim measure, HEFCE earmarked £16 million for supporting start-up research in the former polytechnics. This idea should be built upon by establishing a new element of funding for Research Development, but through the provision of additional funding by the government, rather than by reallocating existing funds. It should be available on application to any researchers in departments with an RAE rating of 1 or 2, or in departments which did not enter into the RAE. It should be available for all areas of research, across all disciplines, and applications should be assessed by established peer review procedures. In order to have a real impact on the quality of research and teaching in our universities, the size of the fund would need to be at least 10 per cent of the total amount currently provided by the three funding councils for research.

2.2  Reducing bureaucracy

  Alongside the creation of seedcorn funding, we favour the postponement of the next RAE. However, if there is to another RAE in 2006, it needs to be in a much more streamlined form, to reduce the burden of bureaucracy and workload on staff.

  One of the ways to achieve this might be to examine whether the RAE is equally relevant to all areas of research. It may be that, at least for as long as underfunding persists, some measure of selectivity is inevitable in certain areas of experimental science in which very expensive instrumentation and installations are required. But do the same arguments apply across all other areas of research, including, for example, theoretical science or philosophy or literary studies? It may be that a close examination of these questions would suggest a much more limited exercise in future, rather than a repeat of the comprehensive assessments that have characterised previous RAEs. Lessons may be learnt from the experience of QAA teaching assessments, whereby universal subject reviews have recently been abandoned in favour of more limited institutional audits.

2.3  Protecting peer review

  Although we are keen to reduce bureaucracy, peer review remains central to any performance-based approach to research funding. Peer review is not perfect, but is widely understood and respected within the academic world. It is also a cost effective way of making use of existing expertise and of spreading knowledge of research activity among academic colleagues.

  We certainly have not come across a better system. The other main performance-based alternative is to use quantitative performance indicators (for example, as in Australia). However, performance indicators such as the volume of external research funds and higher degree research student completions provide only a very rough approximation of research quality. Nor is bibliometric analysis a real substitute to peer review. This is because citations cannot always be used as a proxy for quality (eg some of them are for negative reasons, authors routinely cite their own work, there is a problem of US bias, and there is a time-lag between publication and citation). As a result, we do not believe that citation data is reliable enough for it to become an important element in determining national funding policy.

Association of University Teachers

January 2002


  1.  Aldo Geuna, Dudi Hidayat and Ben Martin, Research allocation and research performance: the assessment of research, SPRU: Brighton, July 1999.

  2.  Tertiary Education Advisory Commission, Shaping the Funding Framework: Fourth Report of the Tertiary Education Advisory Commission, Wellington: New Zealand, November 2001, pp 88-92.

  3.  Ian McNay (1997) The Impact of the 1992 RAE on Institutional and Individual Behaviour in English Higher Education: the evidence from a research project, Centre for Higher Education Management, Anglia Polytechnic University.

  4.  Higher Education Funding Council for England, Review of research (00/37), HEFCE: Bristol, 2000, Table G5, p 62.

  5.  Association of University Teachers, Research Assessment Exercise: response to consultation document RAE 2/97.

  6.  See Times Higher Education Supplement, "Selectivity raises scores across the board", December 14, 2001, p 3.

  7.  The phrase belongs to Professor Lewis Elton, "The UK Research Assessment Exercise: Unintended Consequences", Higher Education Quarterly, Vol 54, No 3 July, 2000, pp 274-283.

  8.  Lee Elliot Major, "Goodbye to all that", Guardian Education, February 20 2001.

  9.  Higher Education Funding Council for England, Characteristics of research active staff, HEFCE Analytical Services Group, March 2000.

  10.  Alan Jenkins, "The Impact of the Research Assessment Exercise on Teaching in Selected Geography Departments in England and Wales', Geography, Vol 80 part 4 (349), 1995, pp 367-374; Ian McNay, "The Paradoxes of Research Assessment Funding", in Changing relationship between higher education and the state, ed. Mary Henkel and Brenda Little, London: Jessica Kingsley Publishers, 1999, pp 191-203.

  11.  Association of University Teachers and the Institute of Education, Academic and academic related staff involvement in the local, regional and national economy, AUT/IoE, 2000.

  12.  Association of University Teachers, Gender differences and activity in the 1996 research assessment exercise (RAE),

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2002
Prepared 7 March 2002