|
UNCORRECTED TRANSCRIPT OF ORAL EVIDENCE To be published as HC 586-i House of COMMONS MINUTES OF EVIDENCE TAKEN BEFORE SCIENCE AND TECHNOLOGY committee
RESEARCH ASSESSMENT EXERCISE: A FOLLOW-UP
Wednesday 19 May 2004 LORD MAY OF OXFORD, PROFESSOR ADRIAN SMITH PROFESSOR IVOR CREWE and PROFESSOR SIR GARETH ROBERTS SIR HOWARD NEWBY and MR RAMA THIRUNAMACHANDRAN Evidence heard in Public Questions 1 - 81
USE OF THE TRANSCRIPT
Oral Evidence Taken before the Science and Technology Committee on Wednesday 19 May 2004 Members present Dr Ian Gibson, in the Chair Paul Farrelly Dr Evan Harris Dr Brian Iddon Mr Robert Key Mr Tony McWalter Bob Spink Dr Desmond Turner ________________ Examination of Witnesses
Witnesses: Lord May of Oxford, a Member of the House of Lords, President, the Royal Society, Professor Adrian Smith, Principal, Queen Mary University of London, and Professor Ivor Crewe, Vice-Chancellor, University of Essex, Universities UK, and Professor Sir Gareth Roberts, President, Wolfson College, Oxford, examined. Q1 Chairman: Can I thank you all for coming along this morning and let me just start by saying to Gareth Roberts, thank you very much indeed for keeping us informed of your inquiry and your investigation into what was going on and filling us in; it has kept us on tap and interested, but certainly we are aware from our work in our constituencies and so on of the feelings around this issue in the academic community and how important it is to many aspects of the higher education policy of the Government, so thank you very much. You have not aged a day! Professor Sir Gareth Roberts: Neither have you, Chairman! Q2 Chairman: Let me start off by asking you, would you call your proposals radical reform in some way? How would you categorise them? Professor Sir Gareth Roberts: I think we can still look forward to 2008/09 and look forward to the Research Assessment Exercise. There are some very, very important changes which are proposed there and the changes which I think really will make a big difference are, for example, the tiered panel structure to make sure that we really do have consistency across the assessment in adjacent subjects and an opportunity to look at standards across those adjacent subjects to make sure that applied research as well as practice-based research is given equal emphasis to pure and basic research, and that was a promise I made to Richard Lambert in fact while he was preparing his review, to make sure that if the quality was there, they should be audited equally. Then probably the biggest reform of all was the quality profile which effectively is still continuous funding rather than the cliff-edge scenarios that we have had with previous RAEs. I am sure you will applaud the thought of introducing submissions based on groups of people. That is the way we do science in particular and many other subjects too, so there is an opportunity here, I think, for some of the people whom we call 'contract researchers' to become involved in the exercise as well. I think there are quite a few substantial improvements and all of those issues that I have mentioned gained at least a seven to one majority in the consultation paper which we put out. Q3 Chairman: So that was your referendum, was it? Professor Sir Gareth Roberts: In a sense it was, but I think one recognises that the RAE is really a historical compromise between the Government's need to introduce performance indicators and accountability on the one hand for that £1.3 billion it spends and then the desire of the academic community to ensure that if their work is going to be assessed, then I think it needs to be based on expert peer review, and again I think the majority there is about 98 per cent of the community in favour of that. Q4 Chairman: I think we agree with the fact that there have been some really big moves forward in incorporating everybody into it. It is a really good move and there is no dodging and weaving and so on which did go on. However, one question which comes through which I think I have to ask all of you is that when you read the evidence which is put before us, there are always people saying, "This is only a sort of halfway house in 2008. There are other things we would like to consider, funding streams and so on", so are we going through this in a half-baked way? Should we look at the whole funding process in universities before we set this up again with perhaps any repercussions which prevent a full analysis of where the money goes and how much is needed? I am saying let's abandon the 2008, I guess, and let's get it right in 2010. Professor Sir Gareth Roberts: Well, I think the key issue now is that we have had 18 years of RAE assessment and we are heading now for an improved assessment process in 2008/09. I would think that assessment to be a much lighter touch and less of a burden to both the academics and the assessors at that time, and I think the secret to that is metrics. You may be aware that the research councils and funding councils are really working very, very hard now on appropriate metrics. Even the HRB and the ESRC are enthused by the prospect, I think, of seeing if they can learn from metrics even in their disciplines. Certainly in 2008/09 I would be shocked if benchmarks from an international point of view were not based on some suitable metrics that we have been working on in the interim. Better still, I would like to see the physical sciences, engineering, hopefully the life sciences and medicine too, those panels guided by the suitable metrics, so there will be more faith, I think, in those metrics which will make the whole assessment process much better. Q5 Chairman: And they will be ready by 2008, you reckon? Professor Sir Gareth Roberts: I believe so. There is a tremendous amount of work going on at the present time. You may not know this, but in Germany, in Australia and in Japan they are thinking of introducing an assessment process based on expert peer review. Clearly they want to learn from the British experience and again the message is coming across that it has transformed, in my belief, the way universities "strategise" about research, but, more than that, we are thinking of, if we can, a lighter-touch assessment and they will learn from that, so they are putting work into metrics as well, and there is a really big initiative to see if we can compare across the patch, I think, the standards in our different countries. Q6 Chairman: Let me invite your compatriots to say something about my challenge to abandon the 2008 until you have all the other feeders in there and do it once and for all with all of the other parts in place. Our evidence suggests that there are other things going on behind the scenes and you have mentioned yourself the funding councils and so on. You hope they will be there in 2008, but let's give it a bit more time and get it right. Lord May, do you have a view? Lord May of Oxford: Yes. Firstly, I would endorse everything that Gareth has said. I agree with you that the report of the work that Gareth has done is really helpful. I would also take a moment to make it clear that it is absolutely necessary that there be two streams of support for research, the project itself and peer review easily handled, and the difficulty is vitally important infrastructure things in central hands. I would agree that if you look around the world at how other places do that, there is a lot to be learned from us. Interestingly, you have just mentioned Japan, Germany and Australia, and in all three of those places, because I have been involved in some of this metric work, I have met with a select committee in Japan, in Australia with the Prime Minister's Science Advisory Council which he chairs and in Germany, so they do look to us to learn. Nonetheless, I would resonate with what you have just said. I think we are committed to the next assessment in 2008 with all the improvements in it, but I would like us now, and this is the position of the Royal Society Council, as we move to 2008 to be thinking more broadly, as you just said, of the whole picture of funding in science because at the moment the greatest worry of the Royal Society Council is not the RAE in its transmogrified form, but the proposals from OST for indirect costs of including even investigator salaries as a direct cost and costing each individual proposal's indirect costs, both of which are proposals which could only have come, I regret to say, from an investigation which was incompetent, did not look at what the consequences are, did not look at other countries and that needs a really hard look much more than this. Lastly, I would say that ultimately the aim also has always to be to ask, however we are doing it, "What behaviour is it going to promote?", so it has got simultaneously to be a just and an appropriately competitive distribution of both the direct and infrastructure indirect funds, but also it has to ask, "Are these well-intentioned actions going to produce perverse consequences?", as the department-based RAE does in its inhibition of co-operative behaviour among departments. Q7 Chairman: So would you postpone the 2008 assessment until all of this visionary analysis takes place? Lord May of Oxford: In a perfect world, I would maybe wish to try, but I think the inertia of this system, it is like trying to turn a tanker around and I think it would cause too much dislocation. There are huge time constants anyhow on the RAE. You are evaluating people on the publications which are derived from research done earlier which derived from funding they got earlier and there are huge time constants in it. Q8 Chairman: Would you not like to know what the funding implications might be though? Lord May of Oxford: I would really like at the same time as we move to 2008 to be thinking now of a fundamental, in-the-round review of what we do next. Q9 Chairman: Do you think that how the research has been used as a metric is perhaps more important than the paper it is published in? Lord May of Oxford: I would say very quickly, and this is a subject in which I have, as it were, professional credentials by this time, that metrics are one of many tools and they have got many faults. One has to be thinking more of the nature of the creative enterprise and, in thinking more widely, I would go further and say that we want to recognise that for some of the infrastructure things, look at what agile universities did with the Business Expansion Scheme when it existed. That produced a lot of creative and sensible building which was market tested and that is how most building goes up in the private and public universities in the United States through tax-free municipal bonds, so a larger look can be very much larger. I get a little worried sometimes at the babble about metrics. Q10 Chairman: Ivor, you have been very quiet and so has Adrian, so how do your members feel? Professor Crewe: First of all, I do endorse what both Gareth Roberts and Rob May have said about the need to examine the research funding of the higher educational sector in the round rather than have separate consultations and examinations of the RAE on the one hand, research council funding on another and sustainability on the third and so on. However, having said that, I do not believe that the UK members would be in favour of postponing the 2008 RAE. To do so would be to set in aspic until 2010 or 2011 the allocation of research funding to universities based on the performance of institutions or parts of institutions in the mid to late-1990s and I cannot believe that that is actually in the interests of the research base of this country, so I think we should go ahead with the 2008 RAE. What universities are looking for is clarity for 2008 about the way in which the RAE will work, clarity in some detail, and some indication of the likely financial return for research performance. What really dismayed the university sector over the last RAE was the retrospective manipulation of the research funding allocation only after the results were known which was very damaging and very disruptive to universities' research financial planning. On metrics, that applies particularly, by which I mean that although institutions would have no objection in principle to a greater use of metrics, particularly if this led to a lighter touch, we will need to know in advance what metrics are going to be used and we would need to be assured that there had been a very thorough appraisal of the validity of these metrics before they were used by the panels, so proceeding with metrics, but with some caution, I think, is what we are looking for. Q11 Bob Spink: Chairman, I am just quite astounded. This is 2004 and we are looking at what is going to happen in 2008, which is a long way off. I am just astounded at the acceptance of inertia where we are looking at picking up a few benchmarks, checking them out, getting them as promises so that we can use them and then changing the system, so I do not see why there should be this acceptance that the system cannot be changed in time for 2008 or even 2008 knocked on the head so that we can move forward with a new system. There seems to be just an acceptance that it cannot happen. Am I missing something here? It really is not rocket science, to be honest. Professor Sir Gareth Roberts: The plan, I think, and you might get this confirmed by Sir Howard Newby shortly, is that we will have the rules of engagement published in about a year's time and by then the individual panels will have mentioned exactly what weight they would place on metrics and we will have that in place by April next year, so there is no reason at all why metrics cannot be used quite usefully in the next RAE. Q12 Dr Iddon: The facts show that there have been disproportionate increases between HEFCE funding and all other funding. I have got some statistics here. In the period 1993/94 financial year up to the 1999/2000 financial year, the facts show that HEFCE funding increased by 25 per cent whilst funding from the research councils and all other sources, including charities, government departments, industry and the EU, increased by 52 per cent, double in fact, which suggests that there is something wrong with the dual-support system. Do you think it has had its day? Professor Sir Gareth Roberts: I am a huge supporter of the dual-support system, as I think ministers are in the OST and DfES. It is the envy of other people in Europe when they know, as they do, about the dual-support system in the UK and they wish they had one. I think it is a nice balance. It has got slightly out of kilter with the Funding Council money not keeping pace with the injection of research money into OST, but that is the basis of this report which Lord May mentioned, the sustainability one. Like him, I think the advice on the principal investigators is not right, but nonetheless, the spirit of trying to get sustainable funding where there is a nicely balanced dual-support system has to be the way forward, I think. Q13 Dr Iddon: I am coming to Lord May separately, so I wonder if I could turn to the other two members of the panel before I do so. Professor Smith: Several of those points for those of us who have to run institutions come back to the same thing; it is the sustainability and the ability to plan and manage over time. I think at this stage any attempt to put back the time of the next RAE would have serious implications for the actual internal dynamics of managing the process. The point you make about the drift apart of the underpinning HEFCE contribution in the dual-support system is important and one of the main, big arguments from UUK is that that component needs to be increased. We do not see it as signalling the end of the dual-support system, but if the two bits get too far out of kilter, the bit which is coming through to the universities for the basic support and the infrastructure, it is increasingly being stressed. The volume of research which is coming through in charities, for example, is exploding, but we are not getting the comparable increase in sustaining the infrastructure, so there is a lot of implication here just for the management of the process, being able to plan and invest ahead. In terms of that planning and forward investment, echoing something which Ivor Crewe said, I think we really do need to know as soon as possible from the Funding Council the broad-brush sense in which they are going to make the funding allocations. Of course we know we cannot be talking absolute sums of money, but, for example, the ratios of financial return in the new system for four stars, three stars, two stars or one star, we do not want the retrospective position again where we suddenly find after the act that two stars and three stars are not being funded or whatever, so I think that kind of clarification upfront is vital for those of us who are called upon to well-manage the system. Q14 Dr Iddon: Lord May, the Royal Society have suggested that the dual-support system does need a fundamental review and you have been rather sceptical of it, as the Royal Society, but we do not appear to have had any alternatives suggested by the Royal Society. I wonder if you could answer both of those points. Lord May of Oxford: Firstly, I confess a grievous incompetence. Using the dual-support system to mean looking at the two strands was a great stupidity on our part and my part in particular. All countries have to have two strands of funding, the direct costs and the infrastructure, and all countries do, but it is just how you handle them. To expand on that a little, in Japan, for example, much of the direct costs are just given out on a per-capita basis, as is the infrastructure money, and the net result is that in terms of what they get in papers, citations and other impact measures for what they spend is one-fifth of what we get. The United States, on the other hand, is hugely complicated and it does both things differently and in a much more diverse way than we do at the moment as our system expands to embrace as large a fraction of young people. I say that partly because I see the ultimate aim for us is to have a tertiary sector which has much of the genuine diversity of the strengths of the US system in that the different institutions within them are diverse and among them are diverse, and one of, to my mind, the greatest problems and unintended consequences of the current RAE is that it is a unidimensional totem pole which focuses too much on the basic research exercise when there are so many other things which are important, not least teaching and the service to local and regional communities. Against that background, I reassert that, as an idealist, and, Ian, I know it was an accusation, not praise, but I enjoyed our exchange the other night here when you called me an idealist, but I am a pragmatic idealist ---- Q15 Chairman: A romantic idealist. Lord May of Oxford: If I were a real romantic, I would wish to take a step back and think fundamentally right now, but, as a realist, I think it is not on and I think that is why we all agree that we are committed to the greatly improved 2008 RAE thing. I hope we are not committed to including PI salaries and stuff, which is a different question, but I hope you will address it, but we ought now to be thinking about the thing in the round. We are only thinking of the balance, as my colleagues here have said, and I agree with everything which has been said, which is unusual, as you know. Q16 Bob Spink: There is not a lot of diversity in assessment routes, is there? Lord May of Oxford: That is why I would wish for this more fundamental review. I must say, I am also worried about too much emphasis on so-called metrics. One of the other perverse consequences you find in common rooms these days as people prepare for these things is that people talk about how many papers in science and nature there have been rather than what was in the papers, and I speak from a position of strength of lots of papers in science and nature, but elsewhere, and it is a curious and, if pursued too long, a very damaging change in the culture. Our focus should be on the ideas and on the judgment of peers about their importance, not where they were published. Q17 Bob Spink: I wonder what the panel thought of this Committee's recommendation that top-rated departments, for instance, should use a metric, that is, their ability to attract external funding in order to save time and costs and probably be more effective? Professor Crewe: I think the difficulty with that proposal is that it assumes that one can assess the quality of research in terms of institutions rather than in terms of the actual people and groups that are doing the research which are units which are very much smaller than an institution. The quality of research should be based on the assessment of those who are doing it and they are typically in groups of six or eight or ten, so an assessment at departmental level may well be appropriate and that is indeed what the Research Assessment Exercise does, but an assessment at an institutional level loses a great deal of information about where the best research has been done. Q18 Bob Spink: But it can make a very effective proxy given that there is a very low turnover at the individual level. Professor Sir Gareth Roberts: I think in subjects like chemistry and so on, it is a good proxy for high-quality research. On the other hand, we have to remember that this exercise covers the social sciences, arts and humanities where really it is very, very difficult to have that sort of metric, in my view. Q19 Bob Spink: Do you think it is fair for me to say that the funding bodies are one-club golfers given that they reject the three-track approach to assessment? Professor Sir Gareth Roberts: Well, the consultation paper gave people huge freedom to suggest whatever they wanted and, first of all, 98 per cent, I think it was, came out in favour of expert peer review rather than a system based on metrics or a historic or a self-assessment approach. When it came to the three-track approach, I think it was fifty-fifty approximately and, by and large, what HEFCE and the other funding councils did was to say, "We will be guided by the community if there is a large majority in favour of a certain change", so I think about 50 per cent of people were actually in favour of the three-track approach. Q20 Bob Spink: Why were there people against it? Professor Sir Gareth Roberts: Well, don't ask me because I proposed that scheme, but I think it was mainly the pride factor. I think the RAE is a big game in town and I think they want to be part of it. I think there are alternative ways of rewarding research and the way people engage with the community and I think these other institutions who are against that approach would have benefited had they gone down the three-track approach because that would have been a much lighter touch for those people. Q21 Bob Spink: I think you are right. Professor Sir Gareth Roberts: Really, Chairman, there are two reasons why people speak against the RAE. One of them we have touched on and Adrian Smith has mentioned the link to funding, not the Research Assessment Exercise process itself, but the link to funding. The other one is whether it has damaged teaching and learning in institutions. I think that we ought between now and 2008/09 to go overboard in trying to stress the true value of teaching and learning. Certainly in my previous university, Sheffield, we promoted people for creative teaching, many of them, not just on the basis of research. I think now that the teaching quality assessments have gone, and they were hugely unpopular, now that they have been replaced by a lighter-touch assessment, the institutional audits by the QAA, I think we ought to be encouraging the QAA really to put at the top of the list, "Are universities rewarding good teaching in terms of promotion, in terms of differential salaries and the like?" Q22 Chairman: But the Higher Education Bill which will presumably go through after the Lords stage now is going to put a lot more functions on universities too in terms of access and so on. Professor Sir Gareth Roberts: Well, I applaud those as well, but I believe ----- Q23 Chairman: The poor people who will have to do it are having their jobs doubled, are they not? Professor Sir Gareth Roberts: Well, I really think you have to play to people's strengths and some people are really very good teachers and what one has to do is reward those people. Maybe one should have a role - this is a quick thought - for those who are not included in an RAE and maybe their institution should receive a 5 per cent premium perhaps to enable them to keep up with their subject by their professional institutions or in other ways so that they disengage from doing the research themselves, but they keep tabs on what other people are doing in research and so on. Q24 Dr Turner: Professor Crewe, you have expressed some views on the international review as part of the process and concern about the possibility that they may be merely tokenistic. What measures would you suggest were taken to ensure that they are not simply there as tokens, but make a real contribution to the process? Professor Crewe: I think the only concern I have got on the part of Universities UK was that the actual use made of non-UK assessors in the last RAE was relatively small. We have certainly no objection in principle at all to asking those from outside the UK system to judge us in the light of international standards, and I think that is right. Probably Sir Howard Newby, whom I know you will be talking to later on, can give you a better answer than I can about what I believe to have been some of the difficulties the Funding Council had in making full use of international assessors, particularly those who knew enough about the UK system to make good judgments. What I think the UK would not want to see would be the placing on panels of international assessors who either did not have, or were not given, the opportunity to inform themselves fully both about the character of the UK higher education system and about the quality of work that particular institutions were submitting to the RAE. It is a matter of practical improvements in the use of international assessors. Q25 Dr Turner: Yes, but I am still not quite clear how you would remedy the current difficulties. Professor Crewe: Well, I understand, and again I do not want to put words in the mouth of Sir Howard, but I understand that the Funding Council has some difficulty in recruiting enough suitable international assessors and then finding a way in which they could be fully involved in the assessment process. I think it would probably be easier for him than for me to explain how they might go about making improvements next time. Professor Sir Gareth Roberts: In my report, I did mention a scheme which does get round this problem. The research councils, and I am thinking in particular of the EPSRC, are just embarking on an international review of physics where they have 12 international experts coming in to spend real time in this country assessing physics. In my report I think I mentioned that it would be sensible for a couple of those people, who really will hopefully by then have understood the research in that subject in this country, perhaps to join the international panels. Another thing which is maybe worth mentioning is that the Germans, I have now had two discussions with them recently, would be more than happy to have a couple of pilot studies with us where we have a joint German and UK team assessing a certain subject or two. Q26 Chairman: But that has not been agreed yet presumably? Professor Sir Gareth Roberts: Well, it has been agreed in private between a couple of consenting adults. Q27 Chairman: But not funded? Professor Sir Gareth Roberts: It could well happen. Lord May of Oxford: I would just say very quickly on the teaching discussion connected with the discussion on encouraging diversity that our current system promotes a mindset which says that either you have a chemistry department which has an active programme of research and produces PhDs or you do not teach chemistry at the university. I think that is crazy. I think we ought to be looking at a much more fundamental way to give universities options that some of the departments maybe do not or do not for all of the faculty-run active research programmes, but still core subjects like chemistry and physics still have to be taught and that is something I would suggest. Q28 Dr Turner: Sir Gareth, how was your suggestion received by HEFCE? Professor Sir Gareth Roberts: The joint German one? Q29 Dr Turner: Yes. Professor Sir Gareth Roberts: Well, they have actually supported a workshop in Oxford in fact to explore the matter further in September, so we will have a meeting there to discuss the pros and cons of it. Q30 Dr Turner: So they have not sanctioned it. How do you see the difficulties in developing metrics which are going to be used in a comparative way, but are going to be very different for different disciplines? What problems do you see there? Professor Sir Gareth Roberts: Well, at the moment there are two studies going on into metrics. One of them is to look at subjects where practice-based research is really quite important and of course this is one of the problems with conventional metrics: how on earth do you assess practice-based research where a piece of research might have an impact on surgery or social policy and so on? So HEFCE has now established, I think, five working groups into nursing, engineering, art and design, management and education really to try and understand more about practice-based research and whether it is possible to evaluate that in some form as a metric, so that is one exercise which is really taking place. The ESRC and the HRB are thinking of metrics which perhaps have not been used in the RAE quite as much in the past, like esteem indicators, people who have maybe chaired important committees who are on or who lead certain editorial boards and so on, so they are really giving a hard look at it. My gut feeling is that they will come to the conclusion that in the arts and humanities particularly there is no substitute for a full-blooded RAE. Q31 Chairman: Ivor Crewe, with gimlet eyes there, is itching to say something. Professor Crewe: I just want to endorse what Gareth Roberts has said. If a panel wants to know what the quality of an authoritative, historical work is, then there is no substitute for reading a book and the same goes for many of the arts and humanities and social sciences. Metrics can provide some supplementary support for the considerations of a panel, but I would be very worried if it was thought that they were some kind of magic wand, some shortcut which could replace the deliberation of peers on the quality of the research which has been submitted. Q32 Dr Turner: The other big problem of course is the assessment by panels of the more traditional research in terms of publications. How do you feel that this actually measures up as a way of assessing the impact of research? Do you think that numbers of citations in the right journals is adequate given that there is not a sufficient lapse of time to assess what impact the research has had in leading to other developments or through into the innovation process or whatever? Do you think that we could do better in evaluating the research impact? Lord May of Oxford: I think they can be a helpful guide, but they really only have meaning at really course levels of aggregation. At the level of individual departments, I think they can be a very frail read. You also have to have a very sophisticated awareness of the hugely different patterns among disciplines. For example, the average, as it were, distance between publication and the modal citation in some areas is 18 months and in other areas it is eight years, and the impact factors average over the past three years incidentally, so there are all sorts of problems. To make international comparisons, I think they are really meaningful, although even there there are problems. At the level of entire institutions, they are tricky, and at the level of small units, like departments, they are indicative, but you cannot use them. Professor Sir Gareth Roberts: I have some data here which I can leave with the Committee which I only received this week from Evidence UK, a company based in Leeds, and they really do show very nicely, I think, just how this country has benefited probably from the RAE as there have been lots of other changes around in the last 15 years too, but if you look at citations in the very best journals in the world, we show a very, very steady improvement. The United States, on the other hand, using the same journals shows a decline and this really is quite interesting and I would be happy to leave those. Lord May of Oxford: In the top one per cent of all cited papers in science, medicine and engineering, scaled against population, and it would be even better if it was scaled against GDP, we were behind the US ten years ago and we are ahead of them now. Professor Sir Gareth Roberts: One of these charts actually shows just that. Lord May of Oxford: That level of aggregation is meaningful. Q33 Dr Turner: The second half of my question was the relationship with the impact both in terms of other research developments, which are built on by other people, the enabling bit of it, and the link through to innovation where we clearly still fall a long way behind the United States. Professor Sir Gareth Roberts: I think this report will come up with extra money for this sort of third-leg funding and of course there needs to be accountability for that, so in looking at that work, I think there will be some Brownie points to be earned by licensing arrangements and spin-outs and I think you will see a dramatic increase and a continuation of the increase in our position. Lord May of Oxford: We need some extra totem poles other than just basic research. Q34 Dr Turner: Do we not need some sort of mechanism, and I do not have it, of assessing the potential importance of research in ten years' time after it has been published or whenever, or it could be 30 years before it leads to something in some cases? Professor Sir Gareth Roberts: I see no reason why you cannot have a paper submitted, maybe one which was submitted in 1989, resubmitted in the next RAE where, on reflection, you can see the impact which that basic research had. That might be one way. Q35 Paul Farrelly: What concerns you most about the terms in which the funding bodies rejected your proposals, that departments do not demonstrate research quality, but they must demonstrate research competence which you define in certain ways as the management and the development of a strategy in departments? Professor Sir Gareth Roberts: Well, as you know, I featured that very strongly because I really do believe that the previous RAEs have neglected the human dimension, the way research students are managed, contract researchers are looked at, probationary staff are managed. The solution here is I think that the funding councils will now include that in their institutional audits every five years when they go on a visit to an institution. That research competence, as we have talked about, will, I think, be examined carefully. There was also this interesting aspect of research into teaching, how good research can impact on teaching, and Sir Graham Davis, in his small working group which is reporting in the next month or two, has been asked to look very carefully at that, so although that recommendation was not accepted as part of the RAE, it is going to be embraced by the funding councils, I believe, in other ways. Q36 Paul Farrelly: Would you envisage that this field of research competence will also be included, looking at how the universities play the system and jockey for position in terms of positioning themselves for the exercise itself? Professor Smith: We have touched several times on the linkage between teaching and research. Bob May has mentioned it and we are now looking at the management of the whole lot. There is something I think we should not duck here, that we are focusing here on the RAE and focusing on strands of research money, but actually the existence of the academic departments on the ground in institutions across science and engineering in the current funding model is dependent on students putting their bums on seats, and there is a fundamental issue there in terms of sustainability which is outwith the particular dynamics of the research funding. Q37 Paul Farrelly: I would just like Professor Roberts to answer that last point as to how your proposal in research competence made recommendations that the exercise look at how universities develop their capacities, and that presumably would include how they might "games-play" as they go through the exercise. Professor Sir Gareth Roberts: Well, in the RAE we did try very hard to avoid the games-playing and in terms of the names submitted for the RAE assessment, I think we have done that. There is a lot more, I think, which we can do for the researchers in academe. Paul Sendry(?) announced, I think, a couple of days ago that I have been asked by the Funders' Forum to chair the Research Careers Initiative and continue work towards developing a new concordat, one which would raise all the things I have just mentioned. I think if that is done in parallel with the QAA making sure that through their institutional audits they look at the way universities approach the whole aspect of teaching and how research links into it, how staff are managed, how research students are managed, I think that is a very legitimate challenge for the groups that do visit these institutions. Q38 Paul Farrelly: To the rest of the panel, I know that as we went through the Higher Education debate recently that the universities resisted, by kicking and screaming, any suggestions that OFFA might become an Ofsted for universities, but in what ways do you think the Research Assessment Exercise could be used for promoting better practices and research competence, as defined by Professor Roberts, within the universities? Professor Crewe: I wonder if I could make two points, one in answer to Mr Farrally's question and then talking about games-playing. In addition to the new remit of institutional audit, I think it also ought to be mentioned that the Funding Council does now expect all universities to provide a human resources strategy in return for the researching and for the rewarding and developing staff initiative, of which there is now quite a substantial tranche of money, and those human resources strategies must cover such issues as the management of contract research staff and also quite specifically the encouragement of high-quality teaching as well as research amongst staff by means of better rewards and better training for teachers, so I think there are two quite separate ways in which the universities are under legitimate pressure and requirements from the Funding Council to ensure that teaching as well as research is rewarded. If I can say something about games-playing, I know that Gareth Roberts tried very hard to produce a system that would reduce games-playing and the Funding Council claims there will be less games-playing in the next RAE, but, as Sir Howard knows because I have said this to him, there will be at least as much, if not more, games-playing in the next Research Assessment Exercise as in this one if the current rules are not changed. There is one provision at the moment which is going to generate a lot of games-playing and that is the decision by the Funding Council, at least for the moment, not to publish any statistics on the proportion of staff being submitted in the next Research Assessment Exercise. Now, if the Funding Council sticks to this decision, what will happen, and I know from my colleagues that it is happening, is that all institutions will be under considerable incentive to be even more selective in the number of staff that they submit in order to improve what might be described as their 'grade point average' and, therefore, to have a higher position in the league tables, which are a very important factor in the recruitment of high-quality staff and the recruitment of postgraduate students, in particular, from overseas. I am hoping that the Funding Council can be persuaded to change its mind on what sounds like a detail, but is actually very important. Q39 Dr Harris: Can I ask Gareth Roberts and/or Lord May that if that is true, is the whole thing not a waste of time because if you have something which is so corrosive to the system that it not only undermines the faith in the system, but actually distorts the results, it may well be better not to do it at all because of the corrosive effect of both those issues, that it is wrong and it is seen to be not right? Professor Sir Gareth Roberts: I think the important thing to mention is that if one introduces a tail this time round of people who are not doing high-quality research, it will not affect the funding one iota. In the previous exercise, if you put a tail in, that could have damaged your grade and you could have gone down from a five star to a four and that would have affected the finances enormously. The fact now that you can put in a tail does not affect your finances, so it is the pride factor, that is all, it is simply a pride factor, and, having had the three-track system rejected of 80 per cent of people which before I had recommended, and that is 80 per cent of people with research in their contracts, I would go now probably for 100 per cent submissions and try and include some post-doctoral people too who are on open-ended contracts who really are seen as contributing and leading the way to research. I do not think it is going to be as divisive as Ivor has suggested and I think there should be some real clarity on it, and it really is important to mention that funding now is not jeopardised. Q40 Dr Harris: Can you understand why they do not publish the proportions then? If there is less of a problem, in your view, can you understand what they possibly have to hide by not publishing the proportions submitted? Professor Sir Gareth Roberts: Well, I would be up for complete transparency. I see no reason at all why one cannot submit everybody. It is a dual-support system and to me anyone who is eligible for applying to a research council for money, it makes sense that they should be included in the RAE. Lord May of Oxford: There is a more fundamental point underlying this. Any system of distributing the money, whether it is the expert peer review direct costs of grants or the infrastructure costs, anything, other than just giving it out on a per-capita basis, needs rules and the rules will govern behaviour. There is no way of avoiding it. There is an interaction between how we do these things and the behaviour we promote. The current system, there is no need to go into the details, but underlying it is one really big problem which is because it evaluates at the level of departments, it does demonstrably inhibit collaboration and indeed one of the things we want is to see a whole diverse, but connected system of research councils, institutes, industry and universities, and this thing that focuses narrowly on the bureaucratic end of it is a problem. The ultimate problem, however, is to recognise that you cannot get away from there being a gain, unless you just give the money out per capita, so you need to think both of what you are trying to achieve and how to do it and then to think very carefully about the unintended consequences of the games we play. One of the obvious consequences of doing this with too much attention is what we see in the universities day by day is that the ratio of administrators to active faculty grows. It is not just the funding councils, but the universities themselves react. If you go back and contrast a university department of 30 years ago when people's main activity was teaching and competing for research grants and look at the amount of bullshit and paperwork which afflicts their daily lives today, it is a disturbing trend. That is why I so resonated what Ian said right at the beginning, which is let's do 2008 because it is difficult to see how we do not, but let's right now, as we move towards that, be looking for a system in the round and putting really fundamental thoughts about it Q41 Mr McWalter: Just following what Evan said and maybe taking a more aggressive line still, it seems to me that whilst people in the system can see that clearly some people who were not research-active have become research-active and some who were research-active have become more research-active, and that is really all very good, I think what Adrian Smith said was vital. I put it to you that the Research Assessment Exercise in the round has actually hugely damaged the system because a huge number of people have pushed their energies and activities into that area to be regarded as a proper academic and all of the other things that academics should be doing are much more marginalised, including many of the very important research activities, like, for instance, doing work on, say, how to get huge changes in developing countries or whatever because that requires four or five sciences to come together and no academic is going to spend their time doing that kind of work because it is intermediate technology or whatever. It seems to me in the end there is a whole host of areas which actually damage the system. Do you agree? Professor Crewe: Chairman, I do not agree with that. I think that not only has the Research Assessment Exercise, despite its evident flaws, made a significant contribution to, and an improvement in, the research base of this country so that we are still second only to the United States in the quality of the research, but I do not believe that it has damaged the quality of the teaching or the interaction with business and the community. If the quality of teaching in British universities had been damaged, it would be very difficult to explain why we have been so successful in recruiting students from overseas and students from the rest of the European Union in the last ten years, and we know that the reason for that is not the research, but it is that UK universities still have an excellent reputation for quality of teaching. Q42 Mr McWalter: And why science departments are closing in droves. Professor Sir Gareth Roberts: I do agree with Ivor Crewe in disagreeing with Tony McWalter, but I think Tony McWalter has a very strong point about how you can help other countries, and I would like to think that in the ten-year science strategy which comes out this July, there will be more co-operation between the Treasury, DfES, OST and the Foreign & Commonwealth Office. Q43 Chairman: But Bob May has made it clear that he wants to see and tends to envisage departments opening again, so is that going to be in the ten-year review, do you think? Professor Sir Gareth Roberts: 'Manpower-planning' is not a term we use in this country anymore, but it does seem ridiculous to me that we have hundreds more people doing sport science than production engineering, for example, not enough people doing chemistry and physics and the reason these departments are closing at the present time is more to do, I think, with the undergraduates' loss of interest in the subjects rather than the RAE. People blame the RAE for everything. Q44 Chairman: But you understand the correlation? Professor Smith: No, I think we should be absolutely clear that really it is not a consequence of the RAEs. It is a consequence of the demand and the funding model which the Funding Council has for teaching income. I think it goes back to other issues mentioned by Gareth Roberts in his report on the supply of scientists and engineers and myself in a mathematics report that we have to get back into schools and look at the engagement of children in science and engineering. Chairman: Well, we agree with that. Mr McWalter: Chairman, could I put on the record that Bob May did nod in approval there. Chairman: Yes, I have been watching the body language and we will publish the results of that independently! Thank you very much, gentlemen, for coming along and setting us off on this exercise and thank you for the work you have put in. Examination of Witnesses Witnesses: Sir Howard Newby, Chief Executive, and Mr Rama Thirunamachandran, Director of Research and Knowledge Transfer, the Higher Education Funding Council for England, examined. Q45 Chairman: Thank you, Sir Howard. I have been watching your body language at the back there and we will publish the number of times you scratched your nose, felt your ears, shrugged your shoulders and smiled; it was very interesting! Thank you very much for coming in, and Rama too. I wanted to start off by asking you if you would like to make a statement and then we can give you some punchy questions. Sir Howard Newby: Well, thank you, Chairman. I did indicate to your colleagues that I might wish to do that and I have got a statement here, but I think in view of the time, the ground which has already been covered and also looking at the body language of the Committee, you would probably prefer to get down to the questioning, so why do we not omit that. Q46 Chairman: We are getting excited about Prime Minister's Question Time! Seriously, you will pick up the issues, I am sure. Let me ask you about this issue and let's be quite specific about this. We said, "This 2008 phenomenon, forget it, get all these other issues sorted out", but what do you feel about that? I guess you would welcome it in a way, would you not, from a work point of view? Sir Howard Newby: Certainly from an administrative point of view, running the RAE reminds me a bit of when I was a student, working on the Christmas post. It is a huge spike in our administrative ---- Q47 Chairman: You did not throw the post away, did you! Sir Howard Newby: No, nor, as far as I know, have we thrown any submissions away, but it is a big administrative burden on us. I think, for the reasons you have just heard, the sector does require a considerable period of preparation and it would like to know what the rules are before it enters the competition, and I think also that we still have an open mind on whether we can reduce the administrative burden by making greater use of metrics, for example, and you might want to come back to that. On the other hand, I have been involved as a Vice-Chancellor in the last two RAEs and each one has always been predicted to be the last one, and the same might be said about the 2008 exercise. I personally find it impossible to conceive of a situation in which research funding, scarce research resources will not be handed out against some measure of performance, some assessment, but I have to say I have a completely open mind about what form that assessment should take beyond 2008. I think the more we can produce an efficient and effective way of developing performance-related research funding without unnecessarily burdening the sector, the better, and I have an open mind on that. Q48 Chairman: In your reply to our report in 2002 you did not accept many of our conclusions, clearly, but there are going to be changes. What do you think of those changes? Sir Howard Newby: I would summarise the present proposals we have put forward as trying to make the existing system of assessment work better, work more consistently across disciplines, and to give proper weight to research activities which are important for the country - and you have already heard some of those mentioned, practice-based research, applied research - and also recognise that different disciplines and different combinations of disciplines, quite legitimately, might be treated in somewhat different ways according to the weightings we give between basic research as evinced in publications, for example, on the one hand, and more practice-based outputs on the other. So we try to make it work better. As you know, Sir Gareth's report did make some more radical recommendations and personally I have always hoped that we might be able to make some more radical changes in the assessment system but, for reasons we may wish to go into, the sector as a whole concluded that it would wish to see improvements in the existing operations of the RAE rather than scrap the RAE altogether. Q49 Chairman: But would you not take up Bob May's position, perhaps, and say that you only have to look at the whole fundamental mechanism of research funding for universities and so on and put that into the pot as well and let's get it right, once and for all, because I think that upsets a lot of people in the academic community who say, "Here we go again. They give it with one hand, take it away with another", and it is very unsettling, but here we have the chance with the ten year review coming up. Sir Howard Newby: I agree very much with what Bob had to say very much, firstly, about the commitment to the dual support system which I think has served this country very well, but again retaining an open mind about how that dual support system might operate in practice. We have had, the last time I looked, nine reviews of science and research funding in this country of various kinds in the last three years, and I think the sector is suffering a little bit from review fatigue on this. If I may, Chairman, I am not trying to be cute when I say this but I would just remind the Committee that in the end the RAE is simply a system by which the Funding Councils, all four of us, calculate the value of the block grant which goes to the university. It is for university managers, quite properly, to take the money we give them and invest where they see fit. They are under no obligation whatsoever to echo the HEFCE funding formula when they distribute resources internally. In fact, when asked, we always advise them not to because we recognise that circumstances vary so much between universities. Q50 Dr Iddon: In your response to our 2002 report on the RAE, you rejected any criticism that we made. If it is not broken, why are you agreeing to fix it now? Sir Howard Newby: Because we have taken on board seriously one or two of the criticisms that were made. If I may single those out, firstly there are issues about whether we were giving sufficient weight to research outputs other than the four publications and academic review journals; whether we had managed the process sufficiently effectively to ensure a degree of consistency across panels and their judgments; and whether or not the way in which the research assessments were being translated into the funding formula meant we were able to sustain sufficiently the very best world class research in this country against increasing international competition. Q51 Dr Iddon: Bearing in mind the comment I made to the previous panel, that there is this big, disproportionate gap in the funding from HEFCE, research councils and all other sources of funding, can you see that the ten year science strategy that Treasury have announced will address that, or not? Sir Howard Newby: It is certainly addressing it. Whether it will close that gap, of course, is dependent upon the spending review decision, and no one knows what the outcome of that will be. Q52 Chairman: What do you think we need? How much? Sir Howard Newby: £900 million. That is the size to which the gap has widened over the period to which Dr Iddon referred. Q53 Dr Harris: On this question we had of games playing, although you say in the new system there will be less of a cliff and therefore less of an incentive for games playing to a certain extent, if anything that might make the tactical nuances more complex, with more energy being put in. Do you recognise that is a problem, firstly, and do you recognise that even if you do not think it is a problem, if it is seen as a problem, it undermines confidence in the system? Sir Howard Newby: Yes, it is a problem, and we do recognise it and I obviously heard with interest what Professor Crewe had to say. I met recently a group of vice chancellors from the '94 Group of universities where we discussed this and let me say on the record that we have not taken a decision finally on how to register the proportion of staff who are submitted. Perhaps I could also say what the difficulties are here. The argument for putting 100 per cent of eligible staff in is that, in our view, this might encourage staff who either are not particularly interested in undertaking research or whose talents lie elsewhere, for example in teaching. They would feel an even greater obligation to undertake research when - and I share some of the sentiments you have just heard - we do more to encourage teaching excellence to put alongside research excellence in our universities -- Q54 Dr Harris: Interrupting on that point briefly, research should then be in their contract. If they are better at teaching and better at doing other things, then research should be in their contract and then they would not count within that 100 per cent. Sir Howard Newby: But, broadly speaking, whilst research is a contractual obligation for staff in the pre 1992 universities, it is not a contractual obligation elsewhere in the sector. We do not want to encourage an unsustainable growth in research volume, because so many more staff who have previously been - I will not say "content" but who have recognised that their talents lie elsewhere feel now an even greater obligation to undertake research. On the other hand, we have the problem articulated by Professor Crewe. I repeat what Sir Gareth said - this has no funding consequences under our proposals but it does have consequences if it is picked up as an indicator by newspapers in the publication of their league tables, because they have in the past taken, and will no doubt continue to take, in looking at their indicators a fraction of staff in a university who have submitted to the RAE, and they do use that as one element in the league tables, and the consequences of that may or may not damage the marketing of institutions when they seek to recruit staff and students, so it is a difficult issue for us. On the other hand, newspaper league tables are not the concern of the Funding Council, but we do recognise that from a university standpoint they do have real consequences. Q55 Dr Harris: We are all afflicted by league tables. You just repeated what Sir Gareth Roberts said, and he was talking about including a tail of some size and I think you are saying that including any number as long as they were relevant would not affect the funding, but how do we know, except by taking your word for it, when you have not said what the funding consequences will be yet as a result of the assessments? In fact, you have not announced that yet so it is very hard for us as a Committee to say that, of course, you are right because we cannot see how that will translate. Sir Howard Newby: What Sir Gareth was pointing out was that whereas, in the past, universities had to make a very difficult decision between, if you like, volume and grade because the funding followed the grade as a single numeric summary grade, they no longer have to make that trade-off because the funding will be based upon the quality of the output across the board and will not be based upon a summative grade, so they do not have to trade off how many staff to put in for fear of losing a particular grade at the end of the day, or vice versa. Q56 Paul Farrelly: But notwithstanding Sir Gareth Roberts' answer, he did say he would be in favour of total transparency. Sir Howard Newby: So are we. Just to pick up Dr Harris' point, we are not hiding anything here. Q57 Dr Harris: It was Ivor Crewe's point. Sir Howard Newby: As I say, the arguments about whether or not we should have 100 per cent entry are evenly balanced and whatever we come down with in the end we certainly are not going to hide anything. The information will be public, and we have been very committed as a Funding Council, and all the funding bodies are, to total transparency in the RAE, and I think on the whole the academic communities welcome that. Q58 Bob Spink: Our 2002 report recommended that top-rated departments should, if they wished, use metrics for their assessment. Why did you not agree with that? Sir Howard Newby: The issue is not whether or not they should be used. They are used, were used last time, and will be used probably more extensively this time because the metrics themselves have got more robust. The issue is whether we should move to an entirely metric-based approach, or whether the metrics should be used as part of a peer review process. The issue there is that, in some subjects, especially in the science and engineering field, metrics are robust, and we certainly intend to guide the panels towards using those metrics as far as they can and also reduce the burden on everybody, but in some other subjects, and the classic ones are the arts and humanities, the metrics are not well developed at all, and are not a very good guide. Q59 Bob Spink: Are you then saying you have six years to work at it, and within those six years you could not correct those metrics you feel are not sufficiently robust? Sir Howard Newby: We can certainly correct them and improve them over that six year period but given the average length of a research project is three to five years there are some huge lags here, so what the committee is looking for now is an indication of what metrics we intend to use, even though it is five years' hence. Q60 Bob Spink: Even though research does take a long time, still each year you will get a number of research projects coming to fruition, so you need not wait for a complete generation to work its way through in order to improve the way that the metrics are generated and reported, and their rigour? Sir Howard Newby: No. I repeat we will do all of that and we will be offering guidance to the panels to use metrics wherever they can, provided we are all satisfied that those metrics are valid. Q61 Chairman: This has not started yet, any of this, has it? Sir Howard Newby: As you have heard, we have been doing some pilot studies on developing the metrics to make sure they can be as robust as possible. Q62 Chairman: But there are no conclusions at all? Mr Thirunamachandran: Let's be clear; there are three sets of metrics which we have used in the past, and will continue to use in the future - research grant information, publication information, and post-graduation research unit information. What we are talking about is ensuring that those metrics are collected as well as they can be and are consistent across the piece in particular cognate subject areas, and then looking at other metrics, particularly in the science, engineering and technology areas, to see whether there are other metrics which could help us even further, and in the case of applied research and practice-based research we think they can. Q63 Bob Spink: The use of benchmarks to compare and manage and control and organise and plan is not unique: business has been doing this for years and years. It is one of the first things you learn at business school. I am surprised that there should be this inertia in academia on this. Sir Howard, you said it was impossible for you to conceive that there would never be any measure of quality in determining the distribution, and you said you were open-minded about what measures should be used after 2008. What I cannot understand is what can we achieve in eight years that we cannot achieve in six, ie for 2008. I do not understand that inertia, that gap of time. Sir Howard Newby: We are talking about very large sums of public money here, over a billion pounds for England alone, plus money for Wales, Scotland and Northern Ireland, and that goes to vice chancellors and their senior management teams to invest in their future research business. They are making investment decisions now where to invest in their research activity, where they think they can get the greatest return in terms of the knowledge created, which I have to say do have lead and lag times of five to ten years. We are now looking at the investments in some key areas of science that were made ten, twenty years ago. So the leads and lags on this are enormous, and you have heard from vice chancellors that what they are looking for is a degree of planning certainty so they can make the investment decisions in a rational way on the understanding that things will not be turned upside down halfway through, and I sympathise with that view. Q64 Bob Spink: You did not like Sir Gareth's three track approach idea. Why? Sir Howard Newby: The argument against it was it was going to produce a degree of complexity from the point of view of institutions which they felt, at this time when we are all concerned about the burden of bureaucracy, was going to be too much. I have to say I do share some sympathy with what lay behind Sir Gareth's proposals. We also wish to arrive at a situation in which those institutions which receive very small amounts of core research funding from us could be treated rather differently from those who receive very large amounts. The smallest amount of money we hand out through our QR, the quality research part of the block grant, is £38,000 a year to one university. My delegated powers as chief executive of the Funding Council are £2 million a year. I could have written them a cheque for £38,000 without putting them through the full rigours of the RAE, but when we came to consult there was a very strong outcry, from even those institutions which were not strong in research, that they had, if you like, almost an entitlement to be entered for the full RAE, and this was really to do with status rather than money. Q65 Chairman: What do you think of OFFA taking over functions that you could well undertake? Sir Howard Newby: I do not like it, Chairman! Q66 Chairman: A waste of money? Sir Howard Newby: I think the policy is absolutely right and one I fully and wholeheartedly support. Whether we need a separate organisation to look after it, both I and my board have severe doubts about. Q67 Paul Farrelly: One of the concerns here and what a lot of the detailed questions have been going on about is the tweaking of the research assessment exercise of 2008, and this is clearly not happening against the background of the status quo. There are two big changes that are coming: one is the view in the White Paper of the future structure of the universities, right or wrong, and we will see how that develops, and, secondly, the repositioning by the universities themselves with the introduction of variable fees. How do you see the research assessment exercise taking into account these potential developments and adapting, and how do you see it adapting to the background changes? Sir Howard Newby: If we were in a situation in which we line-item budgets for universities and say, "You have a little piece of money here for your chemistry research and a little bit of money here for this research and a bit of money here for that teaching and for admin", this would be a very serious matter indeed but to repeat, in the end, when all is said and done, the money goes to the universities as a block grant, and they manage that money in what they judge to be their best interests and they can and do move money around between different headings - between teaching and research, different kinds of research and teaching and so on - and if they did not have that flexibility the higher education system in this country would be very much the poorer. When we look at the impact of the consequences of the Bill in terms of the variable fee regime, they have the flexibility to move money around from us as it is, we do not need to give them more flexibility because they already have it, and since 92 per cent of our allocations go out as a block grant and we only retain 8 per cent for special initiatives, I do not think that the RAE itself needs to be looked at in relation to the provisions of the Bill. Q68 Paul Farrelly: And what about the recommendations in the White Paper regarding the future direction and development of universities, particularly the research and teaching distinction? Sir Howard Newby: There is a general issue here which is faced by all countries who are expanding their higher education systems towards what one might call a more massaged education system, as one might call it, and that is that, at the moment we fund up to 295 institutions, not all of them universities, can we as a nation afford not only to expand and move beyond a 50 per cent participation rate in terms of students but also to expand at a commensurate rate for funding research in those institutions as well, and I do not think there is any country in the world that is doing that. All countries are recognising that, one way or another, as it grows it will become more diverse and some institutions will focus - and I use the word "focus" - on some aspects of the higher education mission more than others. It is not part of my Council's agenda, and I do not believe it is part of the government's, to have so-called "teaching only" universities, but it is part of our agenda to encourage institutions to identify their strengths and focus on their strengths and, as the Committee will be aware, we have also changed the way in which we support universities to attract and retain students from poorer economic backgrounds to meet some of the real costs of doing that, and those institutions which do well at retaining those students now are rewarded quite considerably for doing so. Chairman: That is a familiar line, I think, for some of us. Q69 Mr McWalter: Twice you have portrayed this hands-off HEFCE line which says, "We give you the money as a block grant, do what you like with it", but a vice chancellor with a 5* physics department who has a block of money because that department exists would be a lunatic if he did not then give most of the money that has come in in recognition of that research excellence in order to make sure it also gets him that block of money next time. So the reality is that, however hands off you are, your decisions are in fact mirrored in universities and equally vice chancellors, and this is a point I have made before, are constantly closing departments because they are not bringing them in those blocks of money, hence we are losing chemistry and maths and physics and engineering from university after university. Do you not accept some responsibility for all of that? Sir Howard Newby: Yes, of course I do, and we will come on to the issue of provision of science subjects in a moment, and I am concerned as you are about that, but dealing with the block grant principle first, I said earlier we are a transparent organisation committed to transparency, and therefore it is true that any member of any higher education institution can quite easily work out by going on to our website how our block grant is calculated, and therefore, if you like, what they believe to be their entitlement on the one hand whilst on the other hand we are very clear that we do not line-item university budgets - nor do we wish to, by the way. Now, that does mean that there are institutions which echo the HEFCE funding model internally, but whenever we are asked we say that really they should not; they should use their resources according to their own priorities. Moving on, though, first of all, we share your concern. The fundamental issue here is falling student demand. What supports good research departments is their teaching income, and that teaching income comes on the basis of student recruitment and retention. The figures are that in chemistry the number of departments in England - I obviously cannot speak for Wales and Scotland and I know there is a particular issue in Swansea at the moment - has declined from 59-55 between 1996 and 2003; the number of physics departments has declined from 51-38, and the number of engineering departments from 63-61. The vast bulk of those closures, and there are one or two exceptions, have been in Grade 1 and 2 RAE categorised departments, and they have also been in very small departments, so this is where vice chancellors have been, as I said earlier, taking their investment decisions to invest in areas of growth, and disinvest from areas of -- Q70 Chairman: I thought the RAE did not drive these closures. I cannot remember who said it but it is in my head that somebody said that this morning. Am I wrong? Sir Howard Newby: I do not recall anyone saying it this morning. Q71 Chairman: You are quoting Grade 1s and Grade 2s as a factor. Sir Howard Newby: I am indeed. I cannot recall it being said this morning but if someone has said that to you I am just reporting, if you like, the facts to you. Now, at a national level we might say that 55 chemistry departments and 38 physics departments would be sufficient to service the national need. The problem in my judgment, and this applies to engineering and mathematics as well, by the way, and also to modern languages, is the regional dimension of this. At the regional level, because these closures have been unco-ordinated, unplanned and somewhat random, there are some difficulties. There are no physics or chemistry departments in the eastern region apart from the University of Cambridge and the University of Cambridge, as we know, is not a university which most students can get access to and there are comparable examples elsewhere. To remind the Committee, the Funding Council does not have planning powers but I would certainly accept that we as a nation need to take a more co-ordinated approach to this, so that access to maths, physics, chemistry and engineering provision is available to those students who want it and who can benefit from it, and I am worried at the present time that the rather unco-ordinated nature of some of these closures, even though vice chancellors are acting perfectly rationally, may when you look at the system as a whole produce an effect which is not in the national interest. What do we do about this? In the long term the answer will be to work on the demand side, and we have been running a pilot scheme with the Royal Society of Chemistry in three parts of the country which we are encouraging the Royal Society to roll out which involves university departments, employers working with schools and school children ages 13, 14 onwards to almost sponsor them through 'A' level and beyond. There are some encouraging signs of that and if it does work well we want to look at operating similar schemes with other bodies. Q72 Chairman: Time is running out for that. Sir Howard Newby: It is but I repeat that the fundamental issue is falling demand, a one third decline in chemistry student applications in four years. In the meantime the question is how to sustain provision, especially on a regional level, in the absence of student demand. That is the key issue and I repeat that we have at present no powers to intervene and say to vice chancellors, "You must keep your chemistry department open". We do not have planning powers. Unlike the Learning and Skills Council which has an adequacy provision clause written into its Act we do not, and I think there is something there that should be examined, frankly. Q73 Dr Turner: Can I briefly ask you about the assessment panels? Will you be making extra resources available to them, and will you give each panel a moderator? Sir Howard Newby: The answer to the first question is yes. The word "moderator" is one which is a rather sensitive term to use in other parts of the United Kingdom where it has a certain history, as the Chairman will recognise, but the honest answer to your question is also yes. We do need to ensure greater consistency with the panels, and we also need to make greater use of authoritatively international referees and make more use of their time. Q74 Dr Turner: I was just going to ask about the involvement of user community and overseas members, and how you stop them being seen as tokenistic? Sir Howard Newby: We have to stop them being seen as tokenistic by involving them more in the process than they were the last time. Q75 Dr Turner: How are you going to get consistency across disciplines when using metrics, when you are going to have to have discipline-defined metrics, and you cannot use the same metric across all disciplines? Sir Howard Newby: We are never going to get total consistency. Peer review is a human process and therefore fallible. I would certainly wish to reduce what one might call the standard of deviation between and across panels. I would like us to aim for a good deal of consistency in clusters of disciplines, although recognising that when one tries to compare, say, the judgments made in physics with the judgments made in art history it is very difficult to make an exact comparison. Mr Thirunamachandran: And hence, as Sir Gareth said, the two-tier panel structure is designed to bring cognate subjects together under larger main panels. Q76 Dr Turner: Finally, a main part of previous RAEs has been the publications, the citation index and so on. Is this the only way in which the panels will be assessing the potential impact of research, or will you be looking at wider ranging impacts of that research? Sir Howard Newby: We are looking at wider ranging impacts. Mr Thirunamachandran: Panels in the past and in the future will continue to receive information in what are known as forms(?) 5 and 6, those which were involved in the previous exercise on RAE, to provide a range of information about participation in international activity, whether it be conferences or collaboration with industry. All that information can be provided as part of the submission, so there will be a wider range of information than just publications, grants, and this kind of student data. Q77 Dr Turner: But how will you attempt to measure that, because it is very difficult to foresee the impact in future research developments which are enabled by a piece of research and, likewise, the possibilities in the innovation process? Mr Thirunamachandran: Ultimately I guess it comes down to panel judgment and there is a time lag, as has been said previously, but within those constraints I think the panel will be guided to do their very best to look at the impact it is having on a particular area of policy, or a particular innovation in industry. Q78 Chairman: Lastly, Sir Howard, if you and I were to put ourselves in charge of a department who have been through it, I do not know the rules of the game now and I might not want to play it any more. Why should I play it differently, because I could always stay as a 5 or a 5* without moving, without putting the whole department under this aegis of paperwork and so on. How would you encourage academics to play the game any more? Sir Howard Newby: Having been, like you, a former head of a department, I think the rules of the game of assessment are known, but what is not known is how that translates into cash. Q79 Chairman: Why play it? Sir Howard Newby: I cannot sit here and say "I know how much cash we are going to get to hand out", nor do I know what level of improvement or deterioration in performance there will be as measured by the next RAE, and those are two key variables we do not know, and cannot know in advance, although it would be nice to know how much money we would have to hand out. As you know, what happened last time was that the performance improved to such an extent that the commensurate resources were not made available to fully fund it. Q80 Chairman: Is there any other institutional complex you look at in this country and say, "Gosh, I wish I ran them; I wish I knew we had the money before we started playing this exercise"? Sir Howard Newby: I think this is common across the public sector. Spending reviews these days occur on a two-yearly cycle and a fund for three years with the third year left uncertain because it is going into the next spending review, and whilst all of us, myself included, from the Funding Council wish to have a greater degree of planning certainty further into the future, especially in a long term game like higher education, practically we do not have it and, given the nature of electoral cycles, probably never will. Q81 Chairman: But you do not think the institutions you seek to support and help are different from the Health Service, for example? If I was running a hospital I would know exactly what I needed and I would know how to get it. Sir Howard Newby: In terms of the question you have asked, if I was a manager of a major hospital and if I was a vice chancellor I would also like to know what my reliable funding streams were going to be for more than three years in advance in both cases because, again, of the long term investment decisions one has to make, both in people as well as buildings. Chairman: I think we have heard you say that many times before! Thank you very much for your evidence today. |