Select Committee on Science and Technology Minutes of Evidence

Examination of Witnesses (Questions 1 - 19)




  1. Welcome to our hearing on the Research Assessment Exercise; you are very welcome. I do not think any of you have been here before, and we look forward to your evidence; but perhaps we could start, since we have kept you waiting; there are lots of issues to discuss around education, science, technology, and so on. If you could introduce yourselves, please, I am sure we will get that written into the minutes?
  (Mr Bekhradnia) Certainly, Chairman. Thank you for inviting us, it is always a pleasure to come before a Select Committee. It is particularly apt today, because you know, I think, that the HEFCE Board was considering some of the very issues that are exercising your Committee today.

  2. We time things immaculately.

  (Mr Bekhradnia) Clearly, it is perfectly timed. I am Bahram Bekhradnia. I am Director of Policy at the Higher Education Funding Council for England. John Rushworth, on my left, is the Head of Research Policy at HEFCE; and John Rogers, on my right, alas, shortly to leave us, was Manager of the Research Assessment Exercise, and so knows the Exercise in detail, intimately, if we should get into that sort of discussion.

  3. Thank you very much. Can I start off the questioning then by saying that your Chief Executive, Howard Newby, wrote, indeed, as long ago as last November, that "As the results are compiled, it is becoming apparent that there will be a considerable improvement in performance. No-one should be surprised by this." he said. Now were you surprised by the size of the improvement, and when did this become apparent to you?
  (Mr Bekhradnia) I think, gratified, as much as surprised, would be true. I would have been very surprised and very disappointed if there had been no improvement in the results of the RAE, bearing in mind that the basis for our allocation of research funds and the whole purpose of the RAE is that we allocate money selectively. The RAE is a tool, no more than that. It is a very influential tool, and I do not belittle it by saying that, but it is a tool that enables us to allocate our money selectively. So, with that in mind, back in 1996/97, when we allocated money following the last RAE, there was a substantial injection of additional resource into the top-rated departments. And so, in part, the improvement, in the sense that there was a substantial increase in the proportion of staff in the top-rated departments, was expected; if it had not occurred then something would have been wrong with our funding of research and the whole basis for our funding of research. Beyond that, there is some research which we had seen that indicated that, in terms of citations, for example, UK research in general, but within that in England, was punching very much above its weight; and that differential had increased, we knew that. So that also indicated that there really ought to have been an improvement in the RAE results. So, no, we were not surprised, we were gratified. I think the extent of the improvement was greater than we might have expected, but that there should have been improvement, no, we were not surprised.

  4. Clearly, we will come on to the details of how those improvements came about, and so on, and the quantification of them. The improvements in the results have been said by some, I quote, to be put down to "fiddling, finagling and horse trading." Do you accept that universities' playing of the system has played a part in the improvement in results? It has been equated to the Premier Division; well, perhaps not, perhaps nationwide Division One, in terms of how they have managed to manoeuvre people around to improve their ratings. Do you accept there has been this kind of horse trading going on?
  (Mr Bekhradnia) I think that depends, Chairman, on what you would mean by, I am sorry, I forget, the finagling, fiddling and the—

  5. Transfer deals, if you like?
  (Mr Bekhradnia) We are used to it. It is a sad fact, in this country, perhaps not so much in other parts of the United Kingdom, that when there is improvement we look at ways of explaining it away as being unreal. I think there is good evidence from outside to show why we should have expected an improvement; we also know that research is a very much-better-managed activity in universities now than it has been. One of the things that we did, following the results from the Research Assessment Exercise, was to survey a number of the most research-intensive universities, just to see what management actions they had taken since the last RAE, and there had been, in all cases, substantial programmes of management action, in terms of merging departments, closing departments that were not performing, bringing in new blood; that is management activity, I do not regard that as negative.

  6. But how big was the transfer market, in your opinion, in universities, in the few years leading up to the Research Assessment?
  (Mr Bekhradnia) In terms of numbers—and we can send you the information about this, as it is published; we carried out a review of research 18 months, two years ago, and this was one of the questions we looked at—in terms of numbers, not great. There was no evidence that numbers of staff moving university was affected by the Research Assessment Exercise. It may have had other impacts; it may have led to salary movements that were Research Assessment related, perhaps people being offered incentives not to move, for example; but, in terms of strictly the question that you have asked, which is staff moving between universities, the evidence is that that was not very great. There are some that say that movement of academics in this country is not great enough; in the United States, for example, where they have no Research Assessment Exercise, or anything like it, there is a far higher degree of staff movement between universities, precisely in order to—

  7. That is because they pay them twice as much; they actually bribe them, in the United States, do they not?
  (Mr Bekhradnia) Yes; whatever the mechanism for doing it, they do it. And one of the things that is said about research in this country is actually that there is not enough transfer between institutions, whether due to the RAE or for other reasons.

  8. So you do not think the improvement at all correlates with any kind of transfer market that has been going on; the improvement does not correlate in any way with that?
  (Mr Bekhradnia) Not overall, Chairman; because, if you think about it, if one university is gaining, another must be losing. So, in terms of the overall increase in improvement, I think, certainly not, no.

  9. And you do not think universities have just got slicker and smarter at playing the system?
  (Mr Bekhradnia) Again, I say, it depends on what you mean by playing the system. I think that research is a much better-managed activity now than it has been, I think that it has got a focus, I think that there is more strategy in research activity in universities. And, yes, we do know that after the last Research Assessment Exercise universities did take action—if you look, for example, there are very few 1-rated or 2-rated departments now submitted to the RAE; this is not because of games-playing, of any sort, I do not think, it is because universities have taken management action to merge, to close, to improve areas where they were not so good.

  10. I should declare an interest. I did manage a department, as you would put it, in the last Exercise, and I knew that there was a phenomenon of keeping certain names off the list, so that you put in only a certain percentage of your people as research-active staff. Now that was not putting in the whole staff, that was playing a game, in a sense; I dropped off names of people because they had not published poor quality papers, for example. Now did that go on to a greater extent this time than it did, say, in 1996, it would be?
  (Mr Bekhradnia) John can tell us whether there was a significant change in the percentage of staff submitted; there was a change, but I think certainly it could not have accounted for anything like a significant part of the improvement that was achieved.

Dr Iddon

  11. There is a submission here, in the pile that I got, which refers to materials science; 25 per cent fewer submissions than five years ago, that is in the evidence we have received. Now that is significant?
  (Mr Bekhradnia) Is that fewer staff submitted, or fewer submissions?

  12. Fewer submissions.
  (Mr Bekhradnia) Yes, but then people may be moving out of materials science. They may be merging materials science departments with other departments, and submitting them in a different way. I think that is not strictly comparable. But if you look at the system as a whole, yes, there was a reduction, but I think it was not one that would explain the improvement, in the way that you have described it.
  (Mr Rogers) In terms of numbers of submissions, in 1996 there were 2,894 submissions; in 2001, 2,598. So there was a reduction in the number of submissions. But the number of research academic staff returned was virtually identical in 1996 and 2001. So we have a number of things happening. The weaker submissions, those that were obtaining 1s and 2s, have largely disappeared, because their universities have withdrawn from research in those areas, or through some other management action, such as Bahram has described. So we have seen a reduction in submissions there. We have, in other areas, a number of institutional mergers, which have had an impact on the capability of institutions to submit the same numbers, but we have also seen, in the top-rated submissions, significant growth within departments, which explains a large part of the increase in volume of staff returned in the top grades. So there is a variety of things happening.
  (Mr Bekhradnia) If I may add, that last is precisely the expected improvement; that is selectivity at work. We provided more funds to those departments in 1996, and we would expect universities to use those funds to recruit more staff.

  Chairman: We have a division. We will have a break for ten minutes while we go and cast our democratic vote, on identity cards.

  The Committee suspended from 4.39 pm to 4.47 pm for a division in the House.


  13. We apologise for that. We will return to the subject we were discussing before the division bell. I wanted to ask about the selection of panels, how you select the panels, if there is an indent bias in it in any way; would you like to comment on that first?
  (Mr Bekhradnia) Certainly. Would you like me just to elaborate on one point from your previous line of questioning, Chairman, if I may?

  14. Yes, if you like; you have had time to reconsider, I hope?
  (Mr Bekhradnia) No, no; the advantage of your absence was that I was able to confer with my colleagues. And I can tell you that there was virtually no change in the proportion of the total staff that were submitted, over the whole sector, between 1996 and 2001. Now I cannot comment on materials science, but will gladly provide you, subsequently, if you have got any specific subjects that you are interested in, with specific information, but across the sector as a whole I can tell you that there was a virtually unchanged proportion of staff submitted between the two Exercises. So, in answer to your general interest, I think it is highly implausible that that would have accounted for any part of the improvement between the two Exercises.

  15. Right; well, we will come back to that subject area at a later point.
  (Mr Bekhradnia) On the question of the panel selection, Chairman, again, I will ask John Rogers to comment in more detail, but, in general, let me tell you how we go about this. First of all, I think, we advertised widely, not just in the educational press but more widely.

  16. You advertise in The Guardian, do you?
  (Mr Bekhradnia) And, I think, the Financial Times, and the Times Higher Educational Supplement. It is invidious to mention specific newspapers. But we advertise fairly widely, asking for people to tell us about bodies which might have an interest in research and research assessment, and therefore be interested in nominating people. And we have an extensive list; we had a list of over a thousand, I think, 1,300 bodies that we knew about, that might have an interest in nominating members of panels. We then contacted, we wrote to, all 1,300 bodies, asking them if they would like to make nominations. I think we set out the areas of research of the 69 different panels that we had and the areas that they covered, and asked them, in making nominations, to bear this in mind.

  17. You do not think this is a `clubby' atmosphere, do you?
  (Mr Bekhradnia) We try to avoid that, Chairman. The reason for advertising and for opening up as widely as possible is to avoid that. That would be a danger, and that is the reason why we do it, to avoid that.

  18. Where do you draw the line? If somebody came forward and said, "I did this subject ten years ago," for example, "I remember Mendel's laws but I can't do genetic recombination," are they out?
  (Mr Bekhradnia) I think they ought to be out, probably, if they were not credible people. Because, I think, one of the great things that I will claim for the Research Assessment Exercise is that it has a considerable degree of credibility; and, I think, there, I draw a distinction between that and some of the other processes that are around, and we try to maintain that; one of the important things in that is that the people that are doing the Exercise should be credible. So the nominations come from there, but we have to ensure that the subject is covered in all its dimensions. So the chair of the panel, who is elected by the outgoing panel,—

  19. Elected; is it contested ever, has it ever been contested?
  (Mr Rogers) It was done by a single transferrable vote system within the panel.
  (Mr Bekhradnia) We should not talk about that, in this sort of place, but it is not a `first past the post' arrangement. But the serious point, Chairman, is, yes, we do recognise the danger that you are describing, we do try to open it up as far as we can, and there is a balance between continuity from the last Exercise and the injection of new blood and also a need to cover the full range of subjects that the panel will cover.

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2002
Prepared 7 March 2002