Select Committee on Science and Technology Fifth Special Report


APPENDIX 1

The Government's Response to the House of Commons Science and Technology Committee Report on the Research Assessment Exercise

1. The Committee has produced a timely report dealing with some important issues around the Research Assessment Exercise (RAE). This Government response sets out some general points before responding to the specific recommendations and conclusions. The Higher Education Funding Council for England (HEFCE) will respond separately to the Report. While the RAE is UK­wide and operated on behalf of all the higher education funding bodies, the devolved administrations are responsible for research assessment and the funding of higher education in Scotland, Wales and Northern Ireland. This response therefore focuses on issues relating to England.

The need for excellent research

2. Research is a public good with substantial economic and social benefits. In the UK, as in all advanced countries, the Government funds a key segment of the nation's research effort. Although industry and commerce have a large research commitment, their research investment is, quite rightly, mostly near­market. Business does not tend to engage in speculative basic research, where the returns are often uncertain, long­term, and difficult for one company to capture. Government investment is required to provide the basic new knowledge on which further research can build. A strong publicly funded knowledge base also has benefits in ensuring a supply of highly skilled people with the capability to carry out research in the public and private sectors, to innovate and to make use of new knowledge not just from the UK but from across the world.

Encouraging excellence

3. Government funding for research is via the dual support system. The higher education funding bodies, HEFCE in England, provide funding to institutions. The UK­wide Research Councils provide funding for specific research programmes and projects in their respective areas of interest. The Department for Education and Skills (DfES) sets the overall framework and policy objectives for HEFCE. The Council is then responsible within that framework for devising funding mechanisms that will achieve those policy objectives, and distributing resources provided by DfES to institutions. The DfES also works in partnership with the Department of Trade and Industry, in particular with the Office of Science and Technology (OST) which funds the Research Councils, to ensure a co­ordinated Government approach to the funding of research.

4. Government funding for research will always be limited. It must therefore be appropriately targeted. The first priority must be to sustain and encourage world class research which will support economic growth and improved productivity. The RAE and the quality related funding system are mechanisms developed to pursue this overall objective. Funding via Research Councils also encourages excellence by awarding project funding only to the best research, decided by a process of peer review.

 Achievements of the current system

5. The UK's scientific research is very strong when compared internationally. The UK has the second highest share of publications and citations in the world, and is also second in the world in 15 out of 20 scientific fields, being placed no lower than fifth in any field. The USA has the largest share of publications and citations in each case, but the UK is in first place when it comes to publications and citations per £m spent on research.

6. The system is delivering. Since its development in the late 1980s, the RAE and quality related funding have helped to improve the overall quality and the value for money of research in higher education. As the Committee recognises, the RAE has led to institutions managing their research activity more strategically and has encouraged them to target funds at areas of research excellence.

7. The 2001 RAE results show a higher proportion than ever of UK researchers operating at international levels of excellence. An element of the improvement could be due to institutions getting better at presenting themselves for the RAE. However, citation measures also show a substantial improvement in the UK's research standing since the mid 1990s - the proportion of UK entries in the annual top 1 per cent of most highly cited papers in the world increased from 11 per cent in 1995 to 18 per cent in 2000. This suggests that a good proportion of the improvement shown in the RAE results is genuine improvement in quality. It is welcome that the Committee also acknowledged this in its report.

Current issues

8. The Government agrees that the system does need looking at again.

9. There will always be more demands for research funding than the public purse can afford. The issues around the implementation of the 2001 RAE centred both on the amount of funding available and the way it was distributed. The Government is looking at funding levels for research as part of the current Spending Review. It will also be important to examine how future research assessment and future funding decisions should fit together. However, this cannot become a demand­led budget. Total funding will still be set in Spending Reviews and there will always be a need to allocate limited funding in the way likely to achieve the best results.

10. Given the funding pressures, the Government supports HEFCE's decision to ensure that there was no cut in funding rates, on average, for 5* rated departments. It was most important to ensure that these world class departments were resourced to deliver the leading edge research of which they had shown themselves capable. The funding distribution did however result in lower unit funding for departments with lower ratings despite the additional £30m DfES was able to make available. HEFCE will want to review the distribution in the light of the resources made available to the Council as a result of the Spending Review.

11. There are some further questions to consider on the future development of selectivity. The quality scale now has seven points, of which the three lower grades do not attract HEFCE funding, while more departments than ever are bunched at the top end of the scale where further improvement would not be recognised. The scale needs to provide proper assessment and encouragement for departments at all levels, and allow us to identify and properly fund our world class research.

 12. Some of the other lessons of the RAE will also need to be considered. The Committee's report identifies a number of ways in which the RAE may be influencing behaviour. It is impossible to eliminate side effects and unintended outcomes entirely from an incentive system: the aim has to be to minimise these effects and ensure that the benefits in terms of improved performance outweigh the disadvantages.

Next steps for research funding and assessment

13. The Government continues to believe that research funding should encourage and strengthen excellence, and that higher education institutions should be incentivised to continue to improve the quality of their research following the excellent progress in the last few years. However, the way in which the assessment and funding system operates needs to be revisited. The next steps include:

a.   the 2002 Spending Review settlement which will set out the resources available until 2005­6, and will be informed by the cross­cutting review of science and research

b.   the development of Government plans for the evolution of the dual support system

c.   the development of HEFCE's strategic plan for 2003­08

d.   the review of the RAE.

Response to specific conclusions and recommendations

Accuracy of the RAE results

1.With such a spectacular increase in RAE ratings, it is legitimate to ask whether the improvement is a true reflection of the state of UK academic research and its performance over the last five years. The evidence we have received suggests that most in the science and education communities agree with HEFCE's assertion that it is largely a reflection of reality (paragraph 22).

The Government agrees with the Committee's conclusion that the improvements in research performance as measured by the RAE were largely genuine. Universities and higher education colleges are to be congratulated for the high quality of their research performance, by managing their research strategically and building on their strengths.

2.There is concern about the non­inclusion of researchers. ... Funding should reflect the actual amount of research and its quality over the whole department and not those deemed active. Universities should have no incentive to omit any researchers (paragraph 24).

3.There is concern that by moving researchers between UoAs or splitting and merging departments universities can improve ratings without any improvements in quality (paragraph 25).

4.There is concern that transfers between institutions can distort the RAE results (paragraph 27).

Any quality assurance system has a tendency to encourage those being assessed to present themselves in the best possible way, and the RAE is no different. However, the overall effect of the RAE has been to improve higher education research performance. The RAE has encouraged universities to manage their research strategically and focus on their strengths, bringing real improvements in quality in the sector. They have also focused on staffing and have been made aware of the importance of recruiting, retaining and supporting the top researchers on whom research excellence depends. Government will ensure that these and other issues raised by the Committee are considered in the forthcoming review of the RAE.

5. There are concerns about the way the panels operated and their membership (paragraph 28).

6. We recommend that, in any future RAE, HEFCE provide panel members with more effective administrative support. Ensuring the validity of the results is money well spent (paragraph 29).

The way the panels operate is crucial to the success of the RAE. It is important to ensure that the process is effective at objectively judging the quality of research. That does mean that members should be drawn from a wide range of interests, should have the right knowledge and bring an appropriate mix of perspectives. Strong administrative support is required. It is particularly important to include users such as business people, and to bring an international perspective to bear. The 2001 RAE had more user representatives than the 1996 exercise, and there was a new mechanism for international verification of results, but there may well be scope for still more emphasis on breadth in future.

 7. With the above reservations we accept the widespread view that the RAE ratings reflect an improvement in UK higher education research (paragraph 31).

The Government agrees with this conclusion. The RAE 2001 ratings reflect real improvements in research quality.

Effects of the RAE

8. Most of the evidence we have received has suggested that the RAE has had a beneficial effect on research in the UK (paragraph 32).

The Government agrees that the RAE and quality related research funding have encouraged improved performance.

9. We have heard of concerns that the RAE has imposed large costs and bureaucracy on universities, hampered teaching, distorted research practice, led to neglect of universities' other activities and severely damaged academics' morale (paragraph 33).

10. If HEFCE believes in the value of the RAE, it should establish clearly how much it costs and show why it is worth it. We recommend that, as part of its review of the RAE, HEFCE establish with accuracy the cost of the RAE 2001 and publish costings and an explanation of how these were calculated (paragraph 33).

The Government agrees that there should be no unnecessary bureaucracy in the assessment of research. The aim is to get better research for the public money available, not to waste resources through excessive burdens on institutions. It will be important, as part of the forthcoming review, to look at the cost of the RAE and the extent to which it provides value for money. It will also be important to examine the extent to which the system does influence institutions' behaviour, and to minimise inappropriate effects in future.

11. The RAE has undoubtedly brought benefits but it has also caused collateral damage. It has damaged staff careers and it has distracted universities from their teaching, community and economic development roles. Higher education should encourage excellence in all these areas, not just in research. Universities should be assessed on a balanced score­card (paragraph 59).

If peers assess research we expect them to come out with honest judgements on quality. Those judgements may not always fit with the aspirations of staff, departments or indeed institutions, but that open assessment is important in achieving our research objectives. The RAE has no doubt strongly influenced university management practice and strategic priorities, and has brought a strong focus to research. The Government wants to see excellence in teaching standards, knowledge transfer and widening participation as well. HEFCE is reconsidering its funding model with these aims in mind, and Government will consider the interaction between HEFCE funding and other relevant streams e.g. DTI/OST funding for knowledge transfer.

Research practice

12. Some of the most outstanding achievements in UK science have resulted from long periods of research with no outputs ... We are concerned that the RAE process may discourage long­term research of a highly speculative nature and stifle scientific breakthroughs (paragraph 37).

13. While we accept that publication practice is somewhat different in the humanities, we believe that the shorter period of assessment for the sciences discriminates against scientists involved in long­term research. We suggest that seven years would be a more reasonable period of assessment for the sciences as well as the humanities (paragraph 38).

Long term and speculative work is important. Institutions' RAE related funding does in fact provide them with continuing, entirely flexible funding which can be used to carry out this sort of work on their own initiative - including research which Research Councils and other project sponsors are not willing to fund. The review of the RAE will need to consider how frequently assessment takes place, the eligibility period for different subjects, and to what degree 'research in progress' should be assessed.

14. We recommend that HEFCE ensure that its quality assessment does not discourage or disadvantage interdisciplinary research. Such research offers some of the most fertile ground for innovation and discovery (paragraph 39).

Interdisciplinary research can be particularly valuable and should be encouraged. The RAE process was modified in 2001 to allow for interdisciplinary research to be considered more effectively. It will be important to evaluate the mechanisms used in 2001 carefully in looking to ensure that interdisciplinary research is properly assessed in future.

15. We recommend that HEFCE keep unit boundaries under review to ensure that subjects of increasing importance are fairly assessed (paragraph 40).

The Government agrees with this aim, and the issue will be considered in the review of the RAE.

Morale and careers

16. It is clear that the RAE has had a negative effect on university staff morale. Any future research assessment mechanism must be able to give a fair appraisal of the research without tempting universities to continue the divisive and demoralising practice of excluding some academics from the process (paragraph 41).

At present it is for individual institutions to decide which staff they submit to the RAE. How this should be handled in a future RAE mechanism is a matter to be considered in the forthcoming review.

17. We welcome HEFCE's imminent research project into women in higher education research and recommend that it should analyse RAE data as part of this study. It is vital that women's research careers are not further impeded (paragraph 42).

The Government agrees with this recommendation and looks forward to seeing the results of this project. The Government places a great deal of importance on equal opportunities in HE. Like other employers HEIs are responsible for ensuring they implement equal opportunities legislation and fair employment policies.

National priorities

18. The RAE may not be the primary cause of departmental closures [in science and engineering] but we suspect that it is a contributory factor. Nationally important research that makes a major contribution to the economy must not be destroyed because of trends in student demand (paragraph 44).

Higher education institutions make their own decisions on their internal structure, the courses they offer and the research they carry out. This academic autonomy is fundamental to the HE system and helps ensure that the system is adaptable and independent.

The supply of science and engineering talent is of great importance to the UK. The Government commissioned Sir Gareth Roberts to review this area and is currently considering the recommendations of his recently published report.

19. In our view, it is better to address national research priorities through a funding mechanism rather than by tinkering with the assessment process. HEFCE should protect or enhance key research areas by changing the cost weightings for some UoAs or by introducing ring­fenced funding (paragraph 45).

The Government agrees that national research priorities should not be addressed through the assessment process. The Committee's report refers to the DfES suggestion of further discrimination between departments currently rated 5*. The intention would not be to discriminate according to the importance of the subject area, but rather according to quality. National priorities are currently addressed through other funding routes rather than through HEFCE. The Research Councils are a key mechanism for doing this, for example the Government made an additional £250 million available at the last Spending Review to focus on priorities such as genomics, e­science and basic technology. The cost weightings used by HEFCE are a mechanism to avoid distortion by reflecting the relative costs of research in different subjects, rather than an incentive mechanism.

Neglect of teaching and other university activities

20. The RAE, and the funding decisions based on it, create incentives for universities that could lead to them neglecting other areas of their functions: teaching; community involvement; commercial activity; and research in local or regional significance. This may have major implications for the nature of UK universities (paragraph 46).

21. It is not for the RAE to reward teaching, but there must be a counter­incentive to promote good teaching and encourage good teachers. We believe that there must be financial incentives for improving the quality of teaching but that the burdens and problems of measuring teaching quality are such that funding based on it should be a last resort. HEFCE and universities must work together to provide well­paid and prestigious career positions for academics who are primarily teachers (paragraph 49).

The Government agrees that research excellence is only one focus for higher education institutions. Other Government aims are to widen participation, promote high quality teaching and ensure effective knowledge transfer. Institutions should be encouraged to focus on their strengths, and all higher education institutions will not operate at the same level on each of these areas. HEFCE is reviewing the funding model to provide appropriate financial incentives across the range of Government priorities.

The Government provides three times as much funding for teaching as it does for research via HEFCE. The 2000 Spending Review plans included resources to support increases in academic and non­academic pay: an additional £50 million in 2001­02, rising to £110 million in 2002­3 and £170 million in 2003­04.

22. We are supportive of high­quality teaching in a high­quality research environment and find it hard to see how this can be reconciled with the concept of a teaching-only university (paragraph 51).

23. If the best researchers are concentrated in a small number of departments, we risk losing the next generation of scientists. At a time when the Government is concerned about the supply of scientists in the economy, HEFCE should be encouraging high­quality research wherever there is teaching (paragraph 52).

Teaching does need to be in touch with the forefront of the academic subject but this can be achieved through scholarship as well as through research. The Government's priority for research is to support the best work wherever it is found, and to maintain and develop high quality departments so that they can continue to compete with the best in the world. Within the available resources for research it is vital that a policy of selective funding is pursued. If resources were spread thinly enough to encourage research wherever there is teaching, it would endanger the UK's world class research performance and represent poor value for money from the public investment.

 24. We are concerned that the pressures placed on academics, not least through the RAE, make community involvement less likely (paragraph 53).

Higher education institutions have an important role to play in their communities. Some aspects of this role are encouraged through specific initiatives such as the Active Community Fund and the Higher Education Reach­Out to Business and the Community Fund.

Knowledge transfer

25. We recommend that, in its review of the RAE, HEFCE consider the impact of the RAE on knowledge transfer activity, and investigate whether panels have accorded due status to industrial research outputs. The Government wishes to encourage industrial collaboration and the commercialisation of research and HEFCE must ensure that the RAE does not undermine this (paragraph 54).

The Government is committed to promoting knowledge transfer between the higher education sector and industry, and is aware of criticism that the RAE acts as a disincentive. The definition of research that applies to the RAE includes "work of direct relevance to the needs of industry and commerce" and acceptance of this type of work has developed over the life of the RAE. HEFCE increased the representation of industrialists on panels for the 2001 RAE and emphasised that research products of all types including patents, designs, new products and devices, and commercial and technical reports could be included. DfES will ensure that HEFCE evaluates the results of this in the RAE 2001 submissions and results. There are separate funding streams, particularly the Higher Education Innovation Fund, which provide resources for knowledge transfer. The Cross­cutting Review of Science and Research is considering whether more needs to be done to incentivise HEIs to work with industry.

26.The Government may need to intervene to ensure that research excellence is represented in the regions of the UK, perhaps by encouraging regional networks in important subjects (paragraph 55).

Research excellence is currently represented in all regions and this should continue. Government is encouraging better links between HEFCE, Higher Education Institutions, Regional Development Agencies, and other key regional stakeholders. The Cross­cutting Review of Science and Research is considering what more might be done to promote the regional development of research excellence; a good regional spread is particularly important.

27.Research into matters of local importance can be vital to communities and the economy. If the RAE cannot recognise such work a mechanism needs to be identified that will (paragraph 56).

The Government agrees that such research is vital. The RAE can fund this type of research. The RAE simply assesses quality, it makes no judgement on the nature of the research HEIs should carry out. HEIs decide how to use the money they receive.

 28. HEFCE should monitor levels of investment in infrastructure carefully and if necessary introduce a recurrent funding stream (paragraph 57).

The Government has already demonstrated its commitment to investment in research infrastructure with the £1,750 million made available for HE institutions and other publicly funded research establishments, in partnership with the Wellcome Trust, in the last two Spending Reviews. The Cross­cutting Review of Science and Research is considering research infrastructure as part of its contribution to the spending review.

Funding the RAE

29. We believe that HEFCE was right to use RAE 2001 [to determine the research funding for 2002­03]: if you have a selective mechanism for funding it should reflect the current state of research. But we take issue with the way the cake was cut (paragraph 69).

The Government takes note of the Committee's statement but the priority is to support high quality research departments so that they can maintain their positions of international excellence. The Government therefore agrees with HEFCE's decision to prioritise top rated departments.

30.We recommend that the Government introduce and resource a seedcorn fund to stimulate the development of research in new departments, as part of a strategic framework for research funding (paragraph 71).

The Government recognises the potential benefits of developing research in new departments. There is always a balance to be struck between funding new research and funding research of proven quality. At present the emphasis must remain on properly funding research which is already of world class quality. Future funding decisions will need to be taken in the light of the spending review settlement.

31. We recommend that HEFCE introduce a more sophisticated weighting system which accurately reflects the high costs of research in certain scientific subjects (paragraph 73).

Funding weights should reflect the relative costs of conducting research in different disciplines. The mechanism for doing this is for HEFCE to keep under review. This is one of a number of issues which HEFCE and the Research Councils will work on closely in the future.

Responsibility

32. In their evidence to us, HEFCE seemed to believe that any side effects of the RAE were unfortunate and somehow nothing to do with them. If HEFCE has a mechanism for selective research funding then it must take responsibility for any distortion it causes (paragraph 58).

It is impossible to eliminate side effects and unintended outcomes entirely from an incentive system: the aim has to be to minimise these effects and ensure that the benefits in terms of improved performance outweigh the disadvantages.

 33. While HEFCE cannot be blamed for the level of funding that is available for higher education research, it must bear primary responsibility for the way the RAE funding deficit has been handled (paragraph 74).

34. Some responsibility for the funding decisions must lie with the DfES and with the Minister for Lifelong Learning and Higher Education. ... The RAE and the funding decisions based on it have major repercussions for the higher education system. We find it hard to believe that the Minister is prepared to delegate all that power to an unelected quango. It cannot be in the public interest that she should do so (paragraph 75).

The Government sets the policy objectives for public funding of higher education and provides a steer to HEFCE in the annual remit letter. It is HEFCE's responsibility to put those policies into effect. The RAE can never become a mechanism for determining how much money the Government should spend on research. That is a matter for Ministers and we accept responsibility for funding levels. The separation of responsibilities is designed to preserve academic independence while providing a mechanism to improve the performance of the HE sector in line with the Government's priorities.

35. The DfES must also bear some responsibility for the financial dilemma which HEFCE has been facing. ... We appreciate that the Government has a number of priorities in education, but it must not lose sight of the need to maintain and develop an excellent research base. We welcome the Minister's commitment to fight for a generous settlement for higher education in the Spending Review (paragraph 76).

The Research Assessment Exercise is designed to measure the quality of research, it is not a mechanism for determining the total amount of funding that should be made available. In the light of the 2001 RAE the DfES was able to provide an extra £30m to help fund high quality research in 2002­03. Funding decisions for 2003­04 onwards will be taken as part of the current Spending Review.

36. We fully accept that higher education is the responsibility of DfES, not of DTI, but we would suggest that the Cabinet Minister for Science should take a closer interest in the RAE and in the future of higher education research, since it is vitally important to the future of science and technology in the UK (paragraph 77).

The Cabinet Minister responsible for science regularly takes advice on a range of issues relating to research funding and assessment from the Minister for Science and Innovation, the Chief Scientific Adviser, who is also Head of the DTI's Office of Science & Technology, and the Director­General of the Research Councils. The Chief Scientific Adviser is Chairman of the Science and Engineering Base Co­ordinating Committee, whose remit includes both research assessment and the development of the UK science base.

Under­funding

37. Discussions about the mechanism for the allocation of research funding are largely meaningless unless the under­funding of university research is addressed. ¼There is a strong case for a substantial increase in the HE research budget. This should not be less than the £200 million per year required to fund the RAE 2001 using the formula employed until recently and to restore the project funding/QR ratio to 1993­94 levels. Borne in mind should be the chronic under­funding in university research for much longer than this (paragraph 78).

Funding decisions for 2003­04 onwards will be taken as part of the current Spending Review.

38. We welcome the science and research cross­cutting review and trust that it will spell out clearly for the Treasury the value of science and engineering research and its present parlous state of under­funding ... UK university research is already among the best in the world without the funding it deserves. The Spending Review 2000 brought great benefits for the Science Budget. Now has come the time to put right the imbalance in the dual support system by delivering a significant increase in funding for higher education research (paragraph 79).

As the Committee notes, the UK punches above its weight in terms of its research performance. The work of the Cross­cutting Review is being considered as part of the Spending Review.

Options for the future

39. It is generally agreed that the RAE has achieved all it can in its present form. The question is whether we abandon it completely or whether it could form part of a broader funding mechanism for higher education research (paragraph 80).

The RAE does need to be reviewed. The Committee's report will be useful in that context.

40.We are not persuaded that research assessment should rely entirely on success in obtaining Research Council grants (paragraph 82).

Research assessment does not at present rely on success in obtaining Research Council funding, nor is there any proposal that it should. Quality related research funding is part of the plurality of the present research funding arrangements, which we regard as a strength of the UK system.

41. It is generally agreed that there is a future for the RAE, but not in its current form. We need an RAE with a lighter touch (paragraph 83).

The Government agrees that there should be some form of RAE. A quality mechanism is vital in order to help distribute what amounts to nearly £1 billion of public funding for research in England alone. As part of the Review of the RAE, both the Government and HEFCE will consider whether a lighter touch assessment is appropriate.

 42.We recommend that the RAE should take place every six years, with interim assessment as requested by developing departments or as considered necessary by HEFCE (paragraph 84).

43. We believe that the RAE should continue but only as part of a broader higher education research funding strategy in which its side effects and disadvantages are offset by other mechanisms. We suggest the following model for discussion. HEFCE's research budget could be divided into four sections -

(1)  Funding excellence. Top­rated departments would be exempted from the formal research assessment process if they wish [and funded on their ability to attract external funding].

(2)  Promoting new centres of excellence. Other departments could continue to take part in a research assessment process.

(3)  Developing research capacity. Departments taking part in the research assessment process could apply for development money through a bidding process and would be assessed by subject panels based upon the RAE UoAs.

(4)  Fostering external collaborative research. This fund would support the indirect costs of institutions attracting external project funding [for departments entering the formal assessment process] (paragraph 86).

44.This model of research funding could operate within a broader system of higher education funding which provides incentives for excellence in all areas of universities' activities: teaching, community and economic involvement as well as research. The aim should be to produce a coherent funding system, with a small number of flexible funds (paragraph 88).

This model will be carefully considered by both the Government and HEFCE as the RAE and the research funding system are revisited.

45. No doubt the Funding Councils' review of the RAE will consider a range of options for the future. We await its outcome with great interest, since it will have important implications for the future of science and technology in the UK. An effective funding mechanism for research infrastructure will be crucial if we are to maintain and enhance the UK's research excellence and exploit it successfully. It is essential that DfES, the Funding Councils, the devolved administrations, OST and the Research Councils work closely together to ensure that the funding to the science base is coherent and adequate to maintain the quality of UK research. We shall follow developments closely and, if necessary, report again to the House (paragraph 89).

The Government agrees in the importance of good partnership working to ensure a coherent and effective funding system and the continued development of high quality research.





 
previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2002
Prepared 26 June 2002