Select Committee on Education and Skills Minutes of Evidence

Examination of Witnesses (Questions 1-19)


10 JULY 2006

  Q1 Chairman: I think you are both aware of the great interest there has been since the Chancellor of the Exchequer made a reference to the winding up of the RAE exercise and its replacement. There was some surprise that the Chancellor had made that announcement rather than anybody else, but we will come to that later. Looking at this Committee's work on the funding of higher education and who should pay for it two years ago, it did play some part in the resolution of our discussions over variable fees and all that, which many people think was a very important milestone in the development of higher education in our country. Equally, if we do not get the research side and the funding of research right in this country, again that has serious implications for our university system. All this seems to be a bit of a rush. Somebody said to me at a recent conference that I chaired at the Royal Society—we seem to have most of the Vice Chancellors of the country at it—that this all stems from prior to the Chancellor's statement that three leading vice-chancellors went to see either the Chancellor or other people in the Treasury and seemed to persuade them that we needed some changes fast. Sir Alan, I cannot believe that that was the case, but you know how these stories emerge. Sir Alan, why is there haste about all this? It seems all to be in a bit of a hurry.

  Sir Alan Wilson: In terms of stories about three vice-chancellors, that is something I know nothing of, if it ever took place, so I start from that position.

  Q2  Chairman: So the Vice-Chancellors of Imperial, University College London, and Bristol did not go and see the Chancellor!

  Sir Alan Wilson: Well, nothing is impossible, Chairman, but I have no knowledge of it. That is all I am saying. In terms of haste, the notion that it was all very fast for that kind of reason we would say was not the case, partly because the document that was eventually produced is a follow-up to a document that was published in 2004, the original 10-year framework for investment in science. The policies that were further developed in the budget science paper were really a continuation of the policies that were announced in July 2004. From our perspective it has been continuing work. In terms of the Next Steps paper that was published with the budget—and in a sense this almost answers the question, "why the Chancellor?"—it goes back to the 10-year science framework. I think the Chancellor is anxious, as part of the budget, as I understand it, to have a comprehensive review of progress since the 2004 paper, and research was part of that. From our point of view it is an ongoing process, and we have worked with HEFCE all the way through that period in terms of looking at possible metrics and performance indicators. We have talked to Treasury officials and DTI officials. Much of what was presented in the press about the rush, and certainly the story about when DfES officials told them are simply not true.

  Q3  Chairman: So it is not true that you were surprised in the DfES! Professor Eastwood, would you know whether this was greeted with surprise in HEFCE?

  Professor Eastwood: I think there is a parallel story to the one that Sir Alan has just sketched. After the RAE 2001 the funding councils jointly set up a review of the RAE methodology under Sir Gareth Roberts; and on the basis of the Roberts recommendations, the funding councils agreed substantial changes to the methodology for 2008, including a substantial reliance on metrics in the 2008 exercise. At nearly the same time the decision was taken alongside the RAE in 2008 to run a shadow metrics exercise; that is to say to test in real time an alternative lighter touch methodology for research assessment. Indeed, work was in hand within the funding council, and between the funding council and other bodies, to build that alternative model. So there was a direction of travel here towards a robust RAE in 2008, on the basis of what we might broadly call the Roberts methodology; but alongside that to test and chart a new future for research assessment in the world beyond 2008. To that extent, what was announced at the budget and the announcements around the budget were consistent with that direction of travel.

  Q4  Chairman: Professor Eastwood, was it not the case that at the time of the Gareth Roberts report something like 80% of institutions expressed approval and satisfaction with the peer review and the RAE exercise generally. Was that not the case at the time?

  Professor Eastwood: It is certainly the case that in response to the consultation around Roberts there was a strong preference within the sector to retain a significant element of peer review, and that Roberts did; but alongside that there was the move towards a greater reliance on metrics, and a sense too that the available metrics would continue to develop both in terms of range and in terms of reliability as time moved on. I do not think that the position we are now in is anything more than an evolution—a substantial evolution perhaps but nevertheless an evolution of the position we were in around 2002-2003. That is reflected in the kinds of responses that are beginning to emerge from the sector. Of course, there was a flurry of excitement when the consultation document was published, and one would expect that; but I think there was a serious engagement with the issues raised within the consultation document, and something close to a settled view in the sector that the RAE 2008 is very important—it is very important that we get it right for a whole series of reasons—but that this would be the last RAE "in the current form".

  Q5  Chairman: To be honest about this, it is all about who gets the money, is it not? Whatever system you use, it is about who gets the money to conduct the research. Is it not the fact that the sensitivity is that if you change the rules you may be taking money away from one set of institutions or departments and giving them to others? At the heart of this is there someone in HM Treasury or someone in the higher education world or someone in the Department for Education and Skills saying, "the money is going to the wrong people"? Are they saying that? Have we got to change it?

  Professor Eastwood: From the perspective of the Funding Council, a research assessment does three things. First, it identifies and assesses research quality, which is central to the Funding Council's commitment to fund excellence where it finds it. On the basis of that, and importantly, it has constituted a very important benchmark for the quality of research within UK HE. That matters not just in terms of research and research performance internationally, but also in relation to the branding of UK higher education. Thirdly, as you say, Chairman, it is an exercise which underpins the funding allocations that the Funding Council makes to institutions. We could have a long discussion about what might constitute the right kind of distribution funding, but I have heard nothing in the current debate that suggests that the broad allocation of funding is inappropriate. It needs to be dynamic and it will shift over time. There are a number of rather important debates around the funding of interdisciplinary and multidisciplinary research for example, and also about the resourcing of applied research. There are some areas where some concerns have been expressed, and expressed with some force. It is a large step from that to saying that there is profound dissatisfaction at the funding allocations that are emerging; on the contrary, the Next Step document was rather presuming that the funding allocations would remain broadly as they were.

  Q6  Chairman: Sir Alan, is it not the fact that if there was a discontent about the way research funding was being allocated—we have seen a dramatic change over the years in the number of 5-star and 5 departments; the number has increased very rapidly, and that may because research is so much better or because academics have learnt to play the game rather better—or more academics have learnt to play the game rather better. What is at the heart of this haste? Everybody knew that there was a change in the system going on. Everybody knew that in Australia they were moving from a metrics system and planning to move to a peer-reviewed RAE system—Hong Kong similarly. There seems to be a movement in the other direction. We seem to be going into this metrics area against the flow—is that right?

  Sir Alan Wilson: I think it is against the flow in the sense that we have had twenty years' experience of the research assessment exercise.

  Q7  Chairman: Which everybody thinks is wonderful, and they are copying us!

  Sir Alan Wilson: They think that what has been achieved in this country since 1986 is very impressive. It may be that they have to proceed from the equivalent of a time base that we may have achieved 10 years ago or something of that kind. Not uncontroversially—and in that sense you must be right—there has been a reasonable assumption that a new method of both assessment and allocated funds would be appropriate. In terms of your original question, there has not been a position in the DfES or as far as I know anywhere else saying that certain kinds of universities should get more money and some should get less, because at the government level it has always been about policy. Professor Eastwood has indicated some of the current issues about funding in applied research, funding of interdisciplinary research; but the principle that the best research will be funded wherever it is, is something that underlines all of this. The scale of the exercise—which is why we look for a system that is less bureaucratic—is considerable. It is not just simply that the cost is measured—which is substantial but reasonable in relation to what has been allocated—it is the time and, in a sense, the way that it dominates the development of policies in particular institutions. If there is a simpler way of doing it, there is the possibility, as we said in the consultation paper, that we might be able to move from what is a fantastic platform that has been established to being ever more ambitious in the future.

  Q8  Chairman: So you do not think there is any truth in the assertion that the Chancellor and HM Treasury might have been saying, "we want more applied research, more technology transfer, more team working across departments and across universities; we want to see research much more shaped towards what increases the wealth of the United Kingdom and much more practical outcomes". Do you think that has not been a Treasury view?

  Sir Alan Wilson: It may be a Treasury view in the sense that those concerns have been shared right across the sector and right across different funding agencies. Even the research councils, which you might say are primarily there to support basic research, have had an increasing concern with the mechanisms with which the results of that basic research are applied. I think that many would argue that the distance in time in terms of what used to be called a linear model between basic research and becoming useful is shrinking. I think it is a policy question not just for HM Treasury but for all of us—how come the funding needs of research in the economy as well as blue-skies research and research in all kinds of public interests are balanced. At the end of the day it may go back to funding in another sense; that generous though the research budget is in this country relatively, there is never enough to sustain what everybody would like to sustain.

  Q9  Mr Wilson: This is not about a new way of handling research; this is about the Chancellor saving money, is it not? This is about HM Treasury making a grab for £45 million in savings.

  Sir Alan Wilson: I think that all the evidence in terms of the Chancellor's support of science is that he has been committed to increasing budgets rather than saving money on research. In fact in the last two spending reviews he has actually ring-fenced greater than average increases for research funding, and certainly there is no evidence in any discussions I have had with officials that that situation is changing.

  Q10  Mr Wilson: In 2008 we are all going to see tighter times ahead in education. HM Treasury needs to save money. They are looking forward to where they are going to save it, and this is just one of a number of areas that they are targeting, is it not?

  Sir Alan Wilson: I do not want to anticipate discussions on the comprehensive spending review that will take place in all departments, and certainly within the DfES; and it is a matter of political judgment for our ministers at the end of the day to decide on these relative priorities. I would say again, Chairman, that I have no evidence that in any of the government departments that are party to Next Steps, DfES, HM Treasury, Health and DTI, that anybody wants to do anything but sustain a strong research base and provide the funding for it.

  Q11  Mr Wilson: As the Chairman earlier indicated, there are high levels of satisfaction with the current system, so why does it need to be replaced?

  Sir Alan Wilson: In a sense, as I think I indicated earlier, the big savings for the community are less in terms of money, wherever the money savings are channelled—and they could be channelled into further research—it is the particular way in which it has dominated the time of many academics, and there is a good possibility, I would judge—and I am perhaps making a personal comment, Chairman—that it could increase research productivity.

  Professor Eastwood: The direct costs of RAE 2008 will be of the order of £8 million, the direct costs to the Funding Council. The £45 million is a calculation of indirect costs incurred in institutions in preparing for the RAE. Some of those costs are constant costs, costs associated with research management and performance management and so forth. I think there are almost certainly savings to be made here. The system has, as Sir Alan said, matured over twenty years, and some of those embedded costs can probably be stripped out. I would echo what Sir Alan said: that would be a saving that I see being redirected into the research effort, rather a saving that was stripped out of the HE budget.

  Q12  Mr Wilson: The consultation paper assumes that the RAE should be discontinued, and it seems to be on the basis of widely held views, or what people say. Where is the substance? You yourself said it is time taken by academics. Where is the substance? Where is the evidence for the supposition that you are making?

  Sir Alan Wilson: There is evidence that it is possible in principle to run a simpler system because of the correlations between the data reported in the consultation paper.

  Q13  Mr Wilson: Where do I actually see that evidence?

  Sir Alan Wilson: There are the models that are on the website, where the reference was given in the consultation paper. Indeed, in the Next Steps paper there are two graphs on pages 20 and 21 of chapter 4, which show in aggregate levels certainly some very high levels of correlation. In terms of the feasibility of using something like research income and then other indicators as measures of quality that can then be used for funding allocations, the evidence is there, and quite a number of people have believed for a long time that this would produce a simpler method that would free people for research, rather than run the process as it has been run. As we are all agreeing, none of this is without controversy. Any proposals will be controversial, and in any consultation there will be people who say, "Keep the RAE; it has worked very well"; and there will be others who will say, equally strongly, "the RAE is now in a diminishing returns phase; there is a simpler way of doing it; please let us do that".

  Q14 Mr Wilson: You are right that it is extremely controversial, and for that reason do you not think the Government should have made the case for change a lot more strongly than it has?

  Professor Eastwood: Some of the case was made in Next Steps chapter 4. There is an analysis there, and the consultation document rather presumed on that and did not wish to replicate it. The other thing I would say about the document that was published at the beginning of June is that it is genuinely a consultation document. It offers some illustrations in terms of types of models and in terms of funding outcomes of those models. It is a consultation with the sector, asking the sector to engage with the issues and with the analysis. I think that those involved in this process are fully expecting that the sector will come back, obviously with the critical engagement of the kind that you are suggesting; but also will come back with proposals that will take us forward in maintaining the capacity to assess research quality, to have a sensible framework for distribution of research funding, but to do so in a lighter-touch way than has been possible in the last few years.

  Q15  Mr Wilson: But it is not a consultation that keeps the RAE system on the table; it is a consultation on a metrics basis, is it not? Therefore it is not a consultation at all.

  Professor Eastwood: It is a consultation that asks a certain set of questions around the STEM subjects on the one hand, and the arts and humanities and cognate disciplines on the other; so it does recognise that methodologies for different disciplines may vary. To that extent it recognises that there is a varied disciplinary landscape with which research assessment has to engage. It is also asking whether or not we have the right sorts of metrics, or whether the sorts of methodologies we are envisaging will enable us to make appropriate assessments. You are right that it is not saying, "Do you wish to maintain the RAE in its current form?" However, it is I think inviting serious engagement, and it is also inviting respondents to think about proposals which might further elaborate the sorts of methodologies that are implicit in that document. A number of institutions, not least the research-intensive institutions, are beginning to engage with the consultation in precisely that spirit.

  Q16  Mr Wilson: What are you actually trying to achieve? What is the end game for this metrics-based system? What is it that you want it to do that the system is not doing already?

  Sir Alan Wilson: It will be less bureaucratic; it will cost less; it will take less staff time, but still support excellence; and the correlations that I have talked about suggest that it can still support excellence. It encourages ambition, or facilitates ambition, by taking away people from the formalities of the RAE. It responds to what is becoming an increasingly important interdisciplinary research agenda. It connects to the subject that Professor Eastwood talked about, which is the relative amount of support for applied research or user-led research. That is very important for all of us. The extent to which metrics relate to that should, at the end of the day, be part of the funding formula. That is a policy decision for our ministers at the end of the day. This approach will facilitate the examination of alternatives.

  Q17  Mr Wilson: I understand it is less bureaucratic, it costs less and will support quality, but, as the Chairman said, in Australia that is not what it achieved. Lots of research papers have been produced, but the overall quality diminished, which is why they are switching systems.

  Professor Eastwood: We have made very clear in the documents that we need to have a methodology that is capable of robust assessment in the manner of the RAE. We also commit ourselves in the consultation document to model the likely effects of any change in the system; and indeed that is one of the things we are consulting on as well. We are aware that changes in the assessment methodology and changes in the funding methodology will lead to changes in behaviour. Some of those we might wish to drive; some of them we might wish to inhibit. We are aware of the issues that you raise. In modelling the shadow exercise for 2008, and then in evaluating its likely effects, those kinds of questions will be to the fore.

  Q18  Mr Wilson: I have two questions in one, coming back to the evidence: what evidence do you have that the metrics basis is more efficient than the RAE system; and what evidence do you have that the metrics basis is less expensive?

  Professor Eastwood: The evidence for the latter, that it is less expensive, is that it will enable us to run the RAE without the current elaborate infrastructure of panels. We think it will also mean that the amount of preparation that institutions do for an RAE will be diminished. It will not be eliminated but—

  Q19  Mr Wilson: Do you have any estimates of the savings?

  Professor Eastwood: No, we have not, but then until we have devised the model we will not be in a position to—

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2006
Prepared 8 November 2006