MONDAY 28 OCTOBER 2002
Mr Barry Sheerman, in the Chair
Memoranda submitted by AQA, Edexcel and OCR
Examination of Witnesses
KATHLEEN TATTERSALL, Director-General, AQA; JOHN KERR, Chief Executive, Edexcel; and RON MCLONE, Chief Executive, OCR; examined.
(Ms Tattersall) I think, in the first year of a new examination, inevitably, there is more of a focus on the examination than might be the case in the examination that has been running for some time; but I think we also know that whenever we publish results then there is an interest in those results and quite a public debate about them. In this, the first year of A level, I think when the results were published on 15 August we were all very pleased that the day passed as well as it did, because the focus has to be on the students who have attained the grades in question, and, indeed, my recollection of that day is that there was a welcome for the new examination. I recall The Guardian leader of the day, for example, that there was a welcome for the examination and that things had gone so well. I think that what happened since was that there was clearly some concern, dissatisfaction, on the part of some schools, with the grades which their students attained and a questioning of those grades, and that has led to the re-opening of various issues, some of which were very much firmly in the past, but nevertheless a concentration on those issues, which has led, I think, to where we are today.
(Ms Tattersall) No. Looking at AQA as a board, I believe that the job that we have done in this first year of A level is exactly the same job that we have done in all the previous years of the old A level. And, thinking about it from my own personal perspective, where I have been a chief executive for 20 years, and indeed seen the coming of GCSE, for example, in 1988, the first year of that, and knowing some of the problems that people foresaw at that time, I believe that AQA has done an extremely good job. If you look at what AQA was asked to do, as a result of the Tomlinson inquiry, it was to examine only two of the 1,008 boundaries which we set at A level, and the inquiry, which was very open, very public, very transparent, has reaffirmed the boundaries which I set as a result of looking at the Chair of Examiners' recommendations. So AQA, I believe, can be very proud of its record of bringing in the new A level, and, of course, as a board, we are responsible for something like 45 per cent of the grades awarded in A level this year.
(Ms Tattersall) It is a small world, in that there are three chief executives, as you say, and, of course, we have also got to remember that the system operates in Wales and Northern Ireland, so there are also two other chief executives who are involved. All of us work within the Code of Practice, which is laid down nationally, it is laid down by QCA, drawn up by QCA in consultation with ourselves, and all of us work against the criteria which are determined for A level. I believe that we were all working together to try to establish the same standards across the awarding bodies, as we are charged to do, because three awarding bodies have to ensure that their grades and their awards are in accord with each other. We met over the period of the four years, or so, leading up to the new A levels, on several occasions, there is the Joint Council for General Qualifications, that is the forum in which we meet together, and also with QCA, to try to establish all those difficult technical issues which have to be resolved when the new qualification comes into being. And this, remember, was a qualification which was quite different from the qualification that went before it; here we have a qualification made up of two parts, the AS examination and the A2 examination, AS being a qualification in its own right, and A2 being the second half that makes up the A level. I believe we worked as best we could to try to establish those standards, and it is only really in retrospect that some of these problems now begin to emerge, which at the time were not seen as real issues.
(Dr McLone) We do things slightly differently, that is absolutely true. We have all worked, as Kathleen said, to the same Code of Practice, we have worked to the same procedures, and in the end we all come to the same outcome, in terms of the comparability of the results. We do it slightly differently. Where we have started, we start from looking at what the examiners do first and apply statistical evidence afterwards; not all the boards work in exactly the same way, and therefore it becomes more evident in the way, I suspect, we have done it than perhaps in the others. But I think the important thing is that we do work together in looking at the technical issues, that is absolutely true; but it is the way they have been set up in the context of the whole of the implementation of AS and A2 which I think has led us to where we are now.
(Dr McLone) I would say that we have not had real problems, but we have worked exactly to defining an A level standard, in the same way that OCR and its predecessors always have. We have always worked to getting to the examiner judgements first and then looking at statistical evidence, to make sure that we can compare year on year that we are getting to the right overall standard. I think I do go back to the question of AS and A2; we did not know exactly, all of us, where exactly A2 was. There is a real tension between trying to set boundaries at A2 and yet carrying forward a standard which is not A2, since we do not have any archive evidence at A2, there is nothing of that kind, but we do have to carry forward the A level standard, which is a combination of the AS and the A2. So therefore it has been a tension, in trying to establish all of that. The setting of the standard is actually QCA's job, of course.
(Dr McLone) I think it is possible, in applying the Code of Practice, to be looking for what is the overall standard and trying to define what A2 really means, in a way in which all of us were trying to get to the same place, as Tomlinson said, all of us did our best to get to the same place; if you have not got a definition, and there was no definition written down, as to what you are really trying to get with A2, then I submit that we will be looking to do our best to get there.
(Mr Kerr) I have certainly listened very carefully to what my two colleagues have said, and, in fact, I am in full agreement. In terms of setting the standards, I have one year's experience, and clearly I would not claim that Edexcel has not had its problems in the past; but, for this particular year, I am very confident we set the grades professionally, we set them accurately and we set them in accordance with the Code of Practice.
(Mr Kerr) I think, to answer your first question, what has gone wrong here, clearly, 90,000 students had to wait nearly two months to get their grades confirmed, and clearly that is unacceptable. In terms of my own board, we did not change any of the grade boundaries, we co-operated fully with the Tomlinson inquiry, we thought it was very important that we did co-operate and that there was seen to be a public scrutiny of how the grade boundaries were set. At the end of that, I saw no reason to change any of my grade boundaries.
(Dr McLone) The system was flawed, if I may, and I think we are all trying to operate in a flawed system, that really we need to deal with; and I have to say that, personally, I have great confidence in Ken Boston, in putting forward these new committees, that he is putting forward, to try to right what was not done in the past. Tomlinson and Ken have been very clear about that, and I think that we do need to get to the root of those flaws in the implementation of the system that, in my view, and I think in Mike Tomlinson's view, from what he said, exist.
(Dr McLone) It is a matter of how you look at the way it is done and the way in which you can make the measure. Tomlinson, quite properly, put a measure forward for looking at the way it was done; it could have been looked at in different ways. The way we have done it, which is the way consistent - certainly, it shows more in a system where you are in change. When you have consistently an examination that has been taken year after year and everybody is absolutely sure, with the A2, and if I may just use this chart which I sent to you, if everybody is using something where they know the demand is always the same at A level then they have been consistently arriving at it; when you have a demand at AS, to be advanced by A2, it does matter, in the way you are doing it, how it shows, in other words, the perception of what we have been doing. But I do believe that, when it comes to the outcome at the end, you will see, in the comparability of what we do, and we run comparability studies, that we are actually at very closely the same standard all along.
(Dr McLone) With respect, we changed 18 out of 1,012, which is a very small number. But, yes, we did, and it is a matter of doing it in a different context; we had a different context, we had different people present, we were making judgements. The judgements that we made, on the evidence, and the summer, stand, the judgements that we made were done in a different context at this particular time, and I judged it right to be able to make the amendments I did in the 18 units that I did, but, nonetheless, that is quite a small number.
(Dr McLone) I think we reacted all more or less in the same way. I was about setting standards, just as my colleagues were, for the A level examinations. I think there are a lot of lessons to be learned for the future out of this, and I think that, critically, we need to be looking at the lessons that we have in the way we all managed to do it; we are doing it a slightly different way, I grant that.
(Dr McLone) It could be interpreted so, but I think all the methods that we use are quite proper. I think it is still the case that you could do the statistics, and should perhaps get the statistics first. The key thing is where you select scripts; and I think that this is what we really need to be looking at, in the future, trying to make sure that we all carry out what is a much tighter Code of Practice procedure. The way you select scripts and where you select them depends very much on how much information you put into the system to start with; in a very steady state system, people are very confident about where they might go to select scripts, in a completely new system they are not as confident. Therefore, there are ways in which you have got to get the first set of scripts in which you are actually trying to find out where the boundaries are. We did it, I have to say, by saying to the examiners, "Well, where do we look?" and then we looked at the statistical evidence, where the GCSE performs, there are all sorts of things that you properly do every year. I have to say that you could do it, and we could have done it, by looking first of all at statistical evidence, to say, "Well, let's think about this; if we want a new A2 standard then where will the scripts come from."
(Dr McLone) It is critical.
(Dr McLone) We have a set of procedures laid down, that is absolutely true; what turned out, and the way we have actually operated in the last few years, is that we have looked very closely at what the examiners have said, that is the principal examiner, with not very much, I have to say, statistical evidence applied at that time. We then apply statistical evidence, GCSE performance, we look at performance from year to year, of course, to see whether or not we are agreeing. This year, some subjects, they would obviously work something out to get to the right place, and other subjects have not. My job is to bring all the subjects together, just as it is Kathleen's and John's.
(Dr McLone) What I am saying is that when the examiners are looking at the scripts they are looking at where they should be selecting the scripts from. If, in a new system, we are not sure where the A2 is to be set then they are working in the dark, to some extent, if we do not have some statistical evidence applied; there is no archive, there is nothing of that kind.
(Dr McLone) Technically, it is, because of the 50 per cent weighting. But the problem is that, if you have got a 50 per cent weighting, and, I have to say, we argued very strongly for 40/60, my own experience, from being an academic, is that if you have a part one and a part two, and, many universities, the one I was at was certainly like this, you got the balance right between a part one and a part two one year, and then a harder exam the next year, by a 40/60 weighting. It was changed to a 50/50 weighting. A 50/50 weighting has an impact; of course, it all depends on the demand that is set in question papers, but if you look at a 50/50 weighting and you set one at an AS, which everybody said would be at a lower standard, then, the other one, by definition, you are asking for something that is more demanding. You hope to get question papers that are more demanding, but, if you have got to get the higher demand, that 50/50 weighting, in my view, totally distorted really what we have all been asked to do. Now you could actually look at statistical evidence, to start with, to try to get that distortion out of the way; we look at it afterwards.
(Dr McLone) We argued with QCA and with, well, I suspect we argued with - there was a lot of big four that went on, at the time, I remember, and we certainly argued about 40/60 at that time.
(Ms Tattersall) We all started off by arguing for a 40/60 relationship, that is absolutely true, but as the debate went on other factors came into play, one of them being the points which UCAS proposed to award to AS and to A2, and since AS was half of the other then 50 per cent seemed to be an appropriate percentage. I think we were all asked to, as it were, square the circle, but, I have to say, we did not find it as difficult in AQA as Dr McLone is saying that the problem is in theory; for us, we saw that the AS and the A2 made up the old A level standard, that AS was clearly at a lower standard because it was after one year's work in the sixth form, and that the A2 had somehow to bridge that gap between the full standard and, as it were, the half standard. And so we saw it in terms of the less difficult parts of the old syllabuses being in the AS, and the more difficult parts being in the A2, and the whole being the old A level standard.
(Dr McLone) I must apologise, we have not seen it; (and we have got it open, the e-mail ?).
(Mr Kerr) I think, what Kathleen has said, we did get it right. We are actually not making any apologies for the standards set this year; the standards set, certainly by Edexcel, were correct.
(Ms Tattersall) Let me start by saying that AQA did not respond, as it were, to any pressures, of any kind.
(Ms Tattersall) Let me come back then to the discussions that took place between QCA and the awarding bodies, and they are on record, in two letters, which I think you have, a letter of 22 March and a letter of 19 April; 22 March, myself, and 19 April from Bill Stubbs. And they relate to a meeting that we had on 12 March, when inevitably the issue of standards and the new examination came up; and all of us, as awarding bodies, are charged with the maintenance of standards, year on year, and indeed between ourselves, and the issue is, how do you measure those standards. One measure, you could say, is the percentage of candidates who receive a given award in a given year, the outcomes, I will call them, and the discussion that we had on 12 March focused very much on the outcomes, the expectation being that in 2002 they would be very similar to 2001; that worried many of us, because, clearly, many will see the outcomes only as indicators, not as real examples of standards, and the issue is what are the standards. And that gave rise to the letter that I sent to Bill Stubbs, which was really setting out our position, as awarding bodies, that we judged the standards from the evidence, and the prime evidence is the candidates' work, and the subsidiary evidence is the statistical information that Dr McLone referred to; and we had a letter back from Bill Stubbs, which concurred with that view. Now, as far as AQA was concerned, that clarified the issue, we were all talking the same language; we were not talking about outcomes being the same, we were talking about judging the evidence on the basis of what candidates actually did in the examination.
(Ms Tattersall) We had a further meeting, firstly of ourselves and secondly with QCA, on 26 July, and the reason for that meeting was that it was very clear, certainly from the awards in AQA, that the pattern of the outcomes was going to be very different in 2002 than it had been in 2001. What I was anxious to ascertain was whether this was something peculiar to AQA, or whether it was something which my fellow chief executives were also experiencing in their awards; and so we called a meeting of the boards, we ascertained that we were all experiencing the same sort of pattern of results, and we identified the reasons for that pattern of results. One of the major reasons being that there is a big drop-out rate between the old AS and the full A level, people who had performed to the best of their ability at the AS level and then not gone on to take it at A level. And so, as a result of that meeting, we were very comfortable that the results we were seeing were indeed representing truly the true standards that we were expecting, the carrying forward of standards, and we then shared that information with QCA. There was no pressure from QCA to intervene and change the results after that point.
(Ms Tattersall) I think I wrote to Bill Stubbs following that meeting, again to clarify what we were doing. We were a little worried when QCA had mentioned an inquiry, because that seemed to undermine confidence in results, although all of us recognised that with a new system it is inevitable that people want to see how the system has worked, and therefore an inquiry in the general sense was quite acceptable, but in the specific sense, as to whether the grades were right or not, we thought would really have cast doubt, which would have rebounded very badly on the candidates.
(Ms Tattersall) I do not think it is. Mike Tomlinson is recognising that there were clearly pressures, and all of us operate in a very pressurised society, particularly when results are being published, we have to stand up and be counted; and that, of course, is a pressure, that you have to stand up and be counted, you have to be able to say that the standard of the award in this year is the same as the standard of the award in previous years, regardless of what the statistics actually say. And so that is the sort of pressure that we were all operating in. And, certainly, in terms of specifically bringing in a particular percentage, no, that was not the case, as far as AQA was concerned; we followed our procedures and our practices in the normal way.
(Ms Tattersall) I cannot comment on William Stubbs not being in his job; clearly, that was a matter for the Secretary of State.
(Ms Tattersall) In terms of that particular issue, I did not see that as a pressure to actually bring in awards at a particular level, once we had clarified that we were talking the same language, and we were not actually saying that the outcomes for 2002 had to be exactly the same as the outcomes of 2001.
(Ms Tattersall) I do appreciate that. I can only say that, within the context we work, the main pressure on us is to be able to demonstrate that the standard of our awards is commensurate with the standard of previous awards; and in the first year of an examination that inevitably is difficult, because the syllabuses are different, the structure of the examination is difficult, you do not have the same reference points as you had in the past. But that is the sort of pressure that I would describe, but it is a pressure of which we were very aware, even without QCA saying it.
(Dr McLone) I am sorry if it is vague, but what it is really saying is that the standard that we are required to take forward is that of the whole A level, and that, in fact, is what is in the Code of Practice. The Code of Practice does not mention the AS or the A2; whether it should or not, I think, maybe it ought. But the judgements that are required to be made were not at A level, they were at A2, now, by definition, which is not at A level; there has been a debate as to where it should be, that has been the whole issue about what the whole thing has been about, what should be the A2. And, therefore, the senior examiners, in looking at scripts, were being asked to look at something which was not the exact continuation of the A level, by definition, because of a 50/50 weighting and because of what we have been doing with AS. In my view, that would have been a lot easier if it had been a 40/60 weighting, because they could certainly have continued with a view about what was going on in the past A level and drawn a similar conclusion, the weighting would have taken account of it. I do not think, with a 50/50 weighting, and I think this is pretty clear now, that you could expect that; and the examiners themselves, to be fair, understood that they had got to go for a different level. But there is a tension in that.
(Dr McLone) I think the whole of the implementation scheme was flawed from the start, yes, and I believe, and certainly my board believe, that what we should have been seeing was a much cleaner, crisper definition for all of us of what that actually meant, that tension explained. It is inevitable, I think, and Tomlinson said so in his report, that you are going to get different interpretations, I accept ours looks a different interpretation. I have also to say, what matters is the outcomes, and I believe that, the outcomes, actually, if you take a look at where our boundaries have been set, compared with, say, Kathleen's boundaries, you will probably find them in very much the same place.
(Dr McLone) It is, and I do think that Ken Boston is intending to do so.
(Dr McLone) Most importantly, it would be right for all the students and teachers out there; that is what actually matters. Whether life is easier for us, we are exam boards, it is our job, but it must be right and clear for all the students out there.
(Dr McLone) It has been a big change this year, so the first change we have had such a change to one of our major qualifications, and the first really big change to A level in 50 years, of course, splitting it into two. The issue that I think was important to getting it sorted out, apart from that, was the technicalities all to do with whether there should be a trial or not; there were no pilots, we had some pilots to AS, which helped, I think, with AS, we had one or two, I think, Kathleen did one, we did one, I think John probably did one, maybe, at A2, but no consistent pilot at A2. In the end, of course, what actually happened was the pilot of A2. I think that if we had got some pilot, we had tried to define some archive evidence on which we could base a moving forward, I think that would have been immensely helpful. We could also have done something different, I suspect, in the structure of AS and A2 and whether they were linked, because another problem, again, we wrote about this earlier, a problem about this is we had one qualification, AS, embedded in another one; that is a very difficult thing actually to sort out in the end, too, it is much better if you sort out the two and split them up.
(Ms Tattersall) One of the pitfalls that has not been mentioned is the number of units that formed the qualification. In the early stages, when the qualification was being discussed, we certainly argued for four units, not for six, and, one of the difficulties I think there was, that had there been a unit devoted, as it were, to coursework it would have actually exceeded the limits which at that time were being laid down nationally for coursework. So, in a sense, six became the norm as opposed to four. And that, of course, has brought about other problems, like examiner recruitment, and that is a real issue, in terms of the new qualification. So that, for us, was one of the issues, the number of units, the fragmentation of the curriculum; and I think we have got to remember this is a curriculum problem, not just an examination problem. So we felt that, in some subjects more than in others, for example in English, History, the splitting into that number of units was in itself a problem in curriculum terms.
(Mr Kerr) Having piloted material, having exemplar material, out in the schools, would have made it clearer. I think clearly it was a mistake to launch A2 without going through that; and also I think that Kathleen's view on four units is probably easier for everyone to grasp, rather than six.
(Ms Tattersall) AQA only came into being in 2000, so I am now speaking from a different board, like the NEAB, which was one of the boards that formed AQA, and we argued for many years for a change to the system, in particular actually to have an examination which was at a lower standard than A level following GCSE, because there was such an appalling drop-out of young people between GCSE and A level, with no record at all of their achievements, and we felt that that was such a waste of talent. So, as a board, we piloted what we called the E examination, I think it was Extended, I think that was the name, the Extended examination, which was piloted with many independent schools, because they were the only ones free actually to take an examination which was not a formal qualification. And I think that demonstrates where we stood on the issue of change.
(Dr McLone) Yes, I was, indeed. I do believe that Curriculum 2000, as a curriculum driver, has been a great success, because it has allowed students to move into a broader number of subjects, it has allowed students to develop as they can over 16 to 18; it is a time when students change, to think about all sorts of things that they can suddenly decide. And if they are choosing subjects at 15, as, of course, they have to, and then there is at 17 and they have still got a year to go, and "This really wasn't the subject I wanted to do," this was certainly a real success, it seems to me. So I do not think we should be throwing anything out, we certainly should not be pulling up plants yet again to inspect the roots.
(Dr McLone) I think we need to be just a bit calm about it.
(Dr McLone) Of course, you expect those to have an effect. I have to say that, of course, an awful lot of people were already doing modular A levels, they are not new. So the whole business about whether they actually have dropped the units, or they have dropped out, that has been going on for some time, over half of our A levels before were already modular; so we were not experiencing that massive shift because we were going to modular. The fact that there were five, and that therefore they could drop different subjects, that of course is new, and we expected an increase at E because of that.
(Dr McLone) I do not perceive that we have artificially depressed anything. I do believe that we have been looking very hard as to what this A2 standard would be, and A2 is, indeed, a good deal harder than AS, and I think has been differentially seen, across different schools and different colleges, as to what that really meant. And I think Kathleen and John are right, that if we had had more exemplar materials and more worked out then it would certainly have helped.
(Dr McLone) The boards clearly cannot operate in their own vacuum; obviously, to have exemplar material for a new standard, you have got to do it collectively and you have got to do it all together. Therefore, I perceive that it should be down to the regulator to be driving that forward. Whether it was at anywhere else, I would not like to speculate.
(Dr McLone) Absolutely; completely, yes.
(Dr McLone) We certainly, all of us, were very concerned when this was being introduced, and there are minutes of the joint meetings that we had, that it was being rushed.
(Dr McLone) We said it, I am sure, in meetings we have had with the QCA, but we certainly had it in meetings we had with the Joint Council together; so I am sure we pushed it forward.
(Dr McLone) Whoever is in receipt.
(Dr McLone) I am talking about two years ago.
(Dr McLone) There was, yes.
(Dr McLone) Yes, something like that.
(Dr McLone) I think I would ask my colleagues what they feel about that. I think it is important - - -
(Dr McLone) We do communicate, yes; the question is, the response.
(Dr McLone) What I am saying is that we all perceived that there would be problems; we were working with these problems but we did not conclude any answers.
(Mr Kerr) You asked one question to AQA, you did not ask the rest of the panel.
Mr Chaytor: This is why I am now turning to you, Mr Kerr, and Dr McLone as well.
Chairman: Mr Kerr, we are giving you the opportunity to contribute.
(Mr Kerr) I must interject. In terms of the events that Kathleen outlined, I agree entirely, the 12 March meeting, the letter that the three chief executives wrote, because we were extremely concerned by what was said at the 12 March meeting; the letter of 19 April did not reassure me, I felt the pressure, I am sure the integrity was clearly there but the pressure put on by QCA was inappropriate. That was my evidence to Tomlinson, that has been repeated in the press outside.
(Mr Kerr) The events are the same, I think our interpretations are the same; the pressure was clearly inappropriate. To link grades this year back to legacy A levels was only one factor; the most important factor, from Edexcel's point of view, was the student performance, and to depress students' performance based on Government statistics would be unethical.
(Mr Kerr) No.
(Ms Tattersall) Can I just say, I had forgotten this when I was answering your earlier question, I am sorry; we had a routine meeting, a meeting that was in the diary for a long time, on 6 August, in anticipation of the results. Now by that time, of course, everything is done, dusted, we know where we are, and all we are talking about is what the results are and the sorts of explanations that are available for those results; so there was that third meeting, which, I am sorry about, I had forgotten.
(Ms Tattersall) No. As I say, by then, it is impossible anyway, even if anybody had wanted to, by then, your results are ready to go out, they are ready to go out to UCAS. So the answer is, no, there was not.
(Dr McLone) I think what I am saying is that there have been flaws in the system, which it is QCA's responsibility to deal with, and that that has not been sorted, and that is now clear. What I think I am also saying is, what we need to do is look forward, we have got a new Chief Executive in QCA, I think we have every confidence in that, and we must build that new relationship; and I think that we should look forward, in doing that.
(Dr McLone) I say that what we should be doing is working to a completely new system. I think that QCA have the responsibility to define what it is that we have to do in a new structure like AS and A2, it is their responsibility.
(Dr McLone) I say that we operated, as Tomlinson said, with integrity, in a proper procedure, to deliver what we had to.
(Dr McLone) We certainly would; in our submission to the QCA quinquennial review, we made it quite clear that we thought the QCA was trying to fulfil a whole host of roles, some of which were incompatible. For example, it has an operational role in the Key Stage tests, it has a regulatory role for the awarding bodies, and yet at other times it is making its own awards for tests, for example. It defines operations sometimes very crisply, very precisely, in places where, as awarding bodies, we would say that is not their role; but it is their role to set and define a standard, it is their role to regulate that, it is their role to set that very crisply. That, I would have to say, was not as successful as it should have been. And I think, possibly you were perhaps suggesting it, at the root of it is where its independence is. We believe strongly that we should have an independent, responsible to Parliament, preferably, robust regulator, who will be clear - - -
(Dr McLone) I would not say that we have been frightened of them, at all, but it is quite clear that that has a different impact than if it is independent.
(Dr McLone) I have no view on that.
(Dr McLone) As an issue of principle, yes; but I do not wish to make any comment about individuals.
(Dr McLone) It is our view that the QCA should be independent and should be responsible to Parliament.
(Mr Kerr) Absolutely, coming from a regulatory background, I do believe that an independent, robust regulator, with one caveat, a right of appeal, particularly as it is not envisaged at the moment, answerable to Parliament, would help to restore public confidence.
(Ms Tattersall) In general terms, I agree with what my colleagues are saying here. Certainly, we have argued for a very independent regulatory role for QCA, and we believe that that would be the most accountable and transparent way of demonstrating that. If I could say though, where I disagree somewhat with my colleague is in terms of the discussions which did take place between QCA and ourselves about the big issues of timing, and so on.
(Ms Tattersall) With Dr McLone; on the question of the timing issue, for example, we did press our case very, very strongly to QCA. What we do not know is what QCA advised Government on those issues, because those matters are not made public; and this again comes back to the issue of the independence and the transparency of the process. I am sure QCA will speak for itself on that issue, but we did, as awarding bodies, make very clear that more time was needed for the new qualification, and, in particular, to test out some of the technical matters. As I say, it is then for QCA to advise, in the current terms, Government on what to do, and I am sure QCA will speak for itself on that particular matter.
(Dr McLone) I do not think we are that far apart, Chairman.
(Dr McLone) No, no; sorry about that, if I gave a misinterpretation. I do believe that the pressure was there, and I was trying to say the pressure was there.
(Ms Tattersall) There is not a formal appeal process; the formal relationship is certainly between QCA and ourselves in the Joint Council for General Qualifications, and, I have to say, that is at a policy level, but we also have several working relationships and working groups, of a technical nature, between QCA and ourselves. So there is very regular communication. But, in terms of an appeal process, that is something that we do not have and which we have argued for, and have been told that the only way in which we could make an appeal is through a judicial review.
(Ms Tattersall) No; sorry, with the Joint Council, yes. I misunderstood you.
(Ms Tattersall) It would depend on the issue. If it is an issue to do with the system, where we are acting collectively as individual boards, then that would be a matter, I think, for us to act collectively in that context. If it is a matter which affects an individual board, for example, a matter of accreditation, either the qualification or of the individual organisation itself, that will be for the individual organisation to take up.
(Mr Kerr) (? Edexcel - Inaudible - very clear at) 50/50.
(Dr McLone) No. I do understand where you are coming from; but, no. Our job has been to apply 50/50, and that is what we have done, and that is why I have said that we are looking for A2 to be a considerably higher standard than the old A level, that is part and parcel of the 50/50. My argument was that if we had gone for 40/60, if, then it would have made it easier to deal with in a different way; and I think it would. But that is not affecting the outcome now, we have to deal with 50/50, you are quite right.
(Ms Tattersall) I think, in terms of acting as a regulator of the system, acting as a regulator laying down the rules for the system, the powers would be similar to what they were, but I think more clearly expressed, and I hope more clearly focused, in terms of the way in which QCA activated those powers in respect of the boards. We think that that will be the cleanest and the most transparent way of carrying out those responsibilities.
(Ms Tattersall) Clearly, QCA has had some difficulties, both in terms of not having a Chief Executive and also its Chair, only recently a new Chair being appointed. But I have to say that, in terms of the general working relationship that we have with QCA, at a very practical, logistical, technical level, we have very good relationships and confidence in the people that we deal with. I think, as Ron McLone has said, we now have an opportunity to rebuild relationships at the political, as it were, level with the new Chair and the new Chief Executive. So I have every confidence that we will actually strike a proper relationship with the newly constituted team and, if the QCA does change, with any newly constituted QCA.
(Mr Kerr) I think I would agree with that. We have a new Chief Executive in place now, and that all three boards should work very closely with him to achieve some aims. But if there is an opportunity to redesign QCA, perhaps it is removing its role on the design of qualifications and focus much heavier on the regulation side.
(Dr McLone) I agree with everything that has been said; particularly I agree with Ken Boston, when in his speech at the QCA Annual Conference he talked about the restructuring of QCA that was necessary and the reconstituting of it, and I think we have every confidence in Ken that he will deliver that.
(Mr Kerr) The teachers tell us that they value the choice and innovation that three exam boards bring.
(Mr Kerr) And we would echo that; we are here for educational services to students and to the teachers, and I think we continue as three boards.
(Ms Tattersall) I do not really think the main issues that we are dealing with are problems arising from three boards; what I think it is arising from is the problem of a new system, and, inevitably, in a new system you do get the issues that we have been talking about, about interpretation of standards, and I am quite certain that would occur even if you had the one board dealing with it. Furthermore, the volume of candidates that we have in this country taking qualifications, I think, if you put it all into one board, the risk of things going wrong will be far greater than the risk of things going wrong with three boards.
(Dr McLone) We are, of course, talking about England here, are we not, the English awarding bodies and QCA; but I do not think I have to remind you about that.
(Dr McLone) But I do not have to remind you, I am sure, about what happened in Scotland two years ago, when we had one board and one set of problems; and it is not a matter of whether it is three boards or not, one board can make all the problems. And, in truth, if you have only got one board, you could argue that it might all get hidden.
(Ms Tattersall) I think, by the time the Tomlinson inquiry was set up, the lack of confidence, as it were, in the public perception was such that there needed to be some objective look at what had gone on, and some objective recommendations for action. And, therefore, in that sense, I do think the Tomlinson inquiry was necessary actually to break the deadlock, which we had to, in terms of the public perception and the public confidence in the system. As a board, we were very confident ourselves in the grades that we had awarded, and, indeed, in the interventions that I had made, as a chief executive, which, just to remind the Committee, most of the interventions I made were to lower the grade boundaries, not to raise the grade boundaries, in other words, in favour of the candidates, we were very confident of that, but, nevertheless, we felt it important that we co-operated in full with the Tomlinson inquiry, in order to restore the public confidence that was so necessary.
(Mr Kerr) There is no real other effective mechanism for reviewing grade boundaries. The inquiries by results procedure will look at re-marking only. So this was the only mechanism really to try to demonstrate to the public how the grade boundaries were set, and, as Kathleen said, to diffuse what was clearly a tension out there.
(Dr McLone) I think we all welcomed the fact that we had the Tomlinson inquiry. I think the most important thing that was unfortunate was that the whole expectations had been raised of so many students that we had to do something; and the sad thing, and which we are all concerned about, is how many students have had, as it were, two goes, having their expectations raised and then dashed. That clearly is very sad.
(Ms Tattersall) I think we are facing the situation as it is, namely, QCA as it is; there is no indication, as far as I am aware, that QCA is going to be changed in the near future. And I think it is right that we, as examining boards, work in a proper, professional way with the body which exists to regulate the system, with all its other activities, which we somewhat question. So I do not see, myself, a contradiction between a willingness to make the system work as it is, but also saying, as certainly we did in February, when we made our submission to the quinquennial review, that we would prefer to see a QCA that was totally independent, in the way we have been talking.
(Ms Tattersall) Yes, we would certainly feel that the system could be accounted for and be demonstrated to be more transparent and free of any sort of interference if it were a new system; so the answer is yes.
(Mr Kerr) The answer is, very simply, yes, we would have more confidence with an independent regulator.
(Dr McLone) We certainly would.
(Dr McLone) I think there was deep unhappiness because many people were unsure, just not clear, about what was happening and why it was happening; and I understand that.
(Dr McLone) With due respect, some schools are like that, a good deal of the time; if we take a look at our forecast grades, we would never say, I do not think any of us would say, they have an expert prediction of what the outcomes will be, and that has always been the case.
(Dr McLone) It is, it is; but we have done an analysis of the forecast grades, and some of them are very good and some of them are not so good, and it depends, and that has always been the case. It has been true that there have been some people very, very unhappy; there have been some people, I have to say, who have been very happy. We have had a number of letters saying how well they thought the thing had gone, with us. I go back to the fact that a lot of the students out there have done very well, a lot of the colleges worked very hard, and there was a demand, in some way, with Curriculum 2000, that, as an organisation, you gave commitment to Curriculum 2000. And there is a good deal of evidence, on what we have seen, that those organisations that spent time with their students, worked out how they were going to do it over the two years, put some people in for the first January, then into June, then into January, then into June, have actually done very well; the colleges, I have to say, some of them, in particular, have done very well, because they planned it. Not everybody planned it.
(Dr McLone) There are very few of those. I know there has been a lot in the press about it, but we did an analysis for QCA, in their inquiry, and, as you will have seen in that report, there are actually very few who actually got a U in coursework; and, in fact, in English, which was the subject which was most under review, nobody got a U in coursework who got As in the examination, when you actually analysed the results. So I do not think that there is actually an issue which goes along with what we are talking about.
(Dr McLone) First of all, the most important thing is that we are working with QCA exactly on looking at the main points that we need to do to establish confidence and to give guidance; that we have to do fast, and I know Ken Boston says we have got to do it fast, and we will want to do that. We are establishing many more INSET sessions, to be able to advise teachers and to give teachers help. That sort of thing is very important. It is also done subject by subject; it is not done, as it were, globally, we are giving advice to individual subjects where we know there are issues out there.
(Ms Tattersall) I think the main point, as Ron said, is that we are working with QCA to define more closely the standard that we were talking about earlier in this session; but each of us does run our own INSET meetings, in the case of AQA, we have something like 1,000 meetings lined up in the next two or three months to help teachers to understand better the system. And, of course, we do have now archive material to draw on; so that is a better position than we were in last year, when the INSET material was being provided for the first time. In response though to the first question, if I may, AQA did not have a grading problem, I really must make that clear, and I must make clear that, as I said earlier, we deal with something like 45 per cent of the grades awarded in this country. And so we did not perceive we had a grading problem, we certainly did not have any regrading, arising from the Tomlinson inquiry, and we have many letters from schools, who are saying that they are very content with the service which AQA provided for them and their examination results. So, in a sense, while we are talking about all the problems that have occurred, and quite rightly so, I think one has also got to remember that there are people there who have actually performed in the way that they expected in the exam and been rewarded appropriately for their performance.
(Mr Kerr) Thank you, Chair. I disagreed with your opening comments, where you said the exam boards should keep quiet, exam boards should keep below the surface and should not have a voice. I really do not agree with that. I think that is perhaps one of the lessons we do have to learn from this activity. We are not very good at explaining to people what we do. I am looking at the faces, going round, and there is still a lack of understanding here, there is clearly a lack of understanding on the press bench, exactly what we do, and it is not that difficult. And I do think that certainly both Kathleen and Ron have far greater experience, and that, together, or individually, we can actually restore that public confidence by explaining what we do. That was why I agreed to do the interview with The Times, and I think it is something I wish to continue to do.
(Mr Kerr) We are a large awarding body, of which half of the qualifications are involved with general qualifications, at A levels; the other half are the B-Tech qualifications, the vocational qualifications, which receive no publicity at all, and these are very good, solid qualifications, which we firmly believe in, these are the qualifications that are in demand internationally, they are in demand from employers and from fellow education specialists.
(Mr Kerr) Yes, we are still working on our vision statement; but I think it is really to deliver great qualifications, qualifications that enthuse the learner, the qualifications that teachers find it enjoyable to deliver.
(Mr Kerr) As my colleagues have already stated, it is working with Tomlinson, it is working with QCA, to get the standard communicated better to schools and colleges. It is to enhance the training that is provided; we have already provided training to 40,000 teachers this year, we will probably have to do more. And it is getting our message across, that people can trust the grades that are set by the exam boards, and these are very important qualifications.
(Mr Kerr) I am not adamant at all on that. The exam system is still essentially Victorian, it is a large number of pieces of paper; in our own exam board, it is ten million marks, five million pieces of paper, scripts, in a large warehouse, there is little technology that has been applied there. Certainly, the question for me is investment, who is going to pay for that investment and bring technology in; otherwise, we are going to continue with errors and mistakes, which clearly we will strive to minimise, and it is important that we do so, but there are limited reserves within the exam boards, as charitable organisations.
(Mr Kerr) From Edexcel's point of view, I was very, very pleased with the results this summer, given the publicity surrounding the board earlier in the year.
(Mr Kerr) I was very pleased to be out of the public eye. I was very pleased to see that we delivered the results on time and that we delivered them accurately. So, actually, this whole grading issue came as a bit of a surprise to us. In terms of what I would want to change, piloting of the A2s would have helped enormously.
(Mr Kerr) I think some of the aspects from the B-Tec qualifications are that they are essentially assessed by the teachers, the scripts never leave the college, we have an internal verification system, we have an external verification system, teachers and lecturers have a great deal of confidence in applying these qualifications, and they feel confident and they pass on that confidence to the children.
(Mr Kerr) We get the press we deserve; it is our job to try to educate the press, it is our job to make clear what we do, how we do it, and that is bearing fruit.
(Mr Kerr) Referring again, an independent regulator, a regulator who was respected by the public, could stand up and say, "Don't worry; these are the issues, we can resolve it." But I think also a Government spokesman standing up and saying that they had confidence in the exam system, confidence in the exam results, would have gone a long way to allay press speculation.
(Mr Kerr) I would prefer to see the Government taking forward the lead in promoting the qualifications, and promoting confidence in the qualifications.
(Dr McLone) I think that it is absolutely essential that we have a clear remit in which to operate, given by a regulator. I also believe that what we have in the system that we have got is something, as John has said, which is not transparent, and that we need to move on the examination system we have got in this country to make it more transparent, but also to bring it into the 21st century. That will make it a better job.
(Dr McLone) I do apologise, if I have not explained it properly. What I am saying is that we would have preferred it to be 40 pence, because that would have been a recognition that it was not half an A level. But we worked on it being 50/50, in terms of having to get a balance between the two, yet it is something which is not worth 50 pence but you are having to call it 50 pence; that being so, you have got to have something which really should be 60 pence, and you are going to call that 50 pence. All of that means that you have got this complexity of where you are at; and I am sorry if the arithmetic does not add up, but I think it does.
(Dr McLone) We have to apply 50/50, and, in my view, and I think in lots of other people's, the 50/50 meant the A2 was harder than A level, otherwise it did not stack up.
(Dr McLone) Indeed; and that, I think, is part of the flaw in the system, which we referred to earlier. And, also, if I go back to my analogy with the university world, which I was in, if we had said it was 50/50 we would have been asking too much of the final year students, that is absolutely true, but the 40/60 made that balance work. The fact that you say 50/50 means that you are really asking an awful lot, because you are asking something that is not A level; it is this comparison with what we were doing before which is the problem, because many students in the modular course took these three units in the first year sixth before, yes, and they were A level standard, by definition. Now we have them taking it, and we say, "No, it's not A level standard." That has been the problem; it is a flawed process, and it has been flawed, I think, because we have not had the right definition, and the definition should come from the regulator.
(Dr McLone) The roots of the problem certainly happened then. What needed to be done, in my view, was, over that time, to have recognised that the roots of the problem were going to be difficult. Now we spent a lot of time, of course, in 2001, focusing on AS properly, because we have not run A2; and, properly, I think we got AS right, it was welcome. What we did not do, and what I think everybody feels we should have done, is have some exemplar material, so that everybody understood what A2 was, we did not have it, and if I go back and think about it, collectively, or individually, I think, driven by QCA, we should have had exemplar material.
(Dr McLone) I would not like to say. There was certainly no pressure on the individual awarding, absolutely not; but, in terms of that 50/50 decision, I suspect it was a contributory factor, yes.
(Dr McLone) I think there was a concern within the universities, and, as I said, I come from the sector, I can understand it, there was a concern about what on earth AS was supposed to be; and if it was not valued at 50, I think the understanding of the universities was that it would not have been something that they could value. That must have been, in the end, a political decision, of one sort or another, and I am using 'p' with a small 'p', not a large 'P'. But, nonetheless, that must have had an effect on the final decision about whether it is 40/60 or 50/50; they tend to be decisions that are not made on the assessment structure but on other decisions.
(Dr McLone) I would say it was done directly, myself, it would not have been through our Council; directly to QCA or to Government, I would think.
(Dr McLone) Or. I would not know.
(Dr McLone) We need a lot more transparency for everybody. I think the business about whether the AS will count as one point, or not, was something which certainly did not get thought through alongside what that would mean if you had put it in terms of assessment structure; and, of course, there is an argument which says the two should be divorced anyway.
(Dr McLone) I do understand the question, and I think there is a tension; there is a tension between picking up plants, as I said, and inspecting the roots, because you want to embed something that is already going. There are flaws which I believe will be put right through Ken Boston's procedures over the next few months; those are the flaws, and that is the way it must be put right.
(Dr McLone) By next year, I think we will put it right, by talking to Ken Boston's task group and Ken Boston's Programme Board and the arrangements he is getting right, and Tomlinson's inquiry, I think we will be looking to get things out that can match next year and get it on the road. I think there are longer-term issues that we will have to address, like six/four units, because, eventually, there is really too much assessment going on, and it is overburdening our teachers and students.
(Ms Tattersall) I think we are externally overexamining our students, and that, I think, is where the problem lies; and certainly there has been a trend to external examining, over the last 14, 15 years, which actually has swept away some very good coursework-based examinations. So I distinguish between assessing of students and externally examining our students, and I think the balance, as it were, has tipped too far to external assessment, and, as I said earlier, that brings with it some problems, such as recruiting examiners to fulfil our requirements. So I would prefer to see the pendulum swing somewhat back, to enable teacher assessment to take place, but, in order for that to happen, I think we would have to recognise that there would have to be a lot of training of teachers, for a start, in assessment methods, there would have to be very robust systems of moderation, and there would have to be, I think, a turnaround of public perception of the value of teacher assessment, because that was where it went wrong 14 or 15 years ago, when people really started questioning whether that was valuable.
(Ms Tattersall) And the important thing, therefore, is moderation; and, therefore, as an examination board, we would certainly welcome more emphasis on the teacher involvement in assessment, but it would have to be in a context where there was a very clear framework of moderation provided by some external body, of which we would be one of them, we hope, and we would provide the exemplar material for teachers, we would do the training of teachers, and we would moderate samples of the work which they assessed.
(Ms Tattersall) I think what Ron McLone was talking about was the information that we provide to the awarding meetings when they are making their decisions about grade boundaries. And we provide, at the outset, both the candidates' scripts and some statistical information which will help the awarders come to a judgement about those scripts; and that statistical information, as Ron McLone said, is GCSE mean grades, it is AS performance of the candidates who are being judged on the A2, it is information in normal circumstances relating to the previous examination, in terms of how candidates performed, and we provide that from the outset. So we provide parameters, which show very clearly what the impacts of judgements are going to be and how they might compare with, let us say, the GCSE mean grade data. We believe that interaction between the evidence is important for people actually to understand the scripts themselves; because there is an awful lot of research evidence to show that if you simply present scripts in a vacuum then people are not very sure exactly where they relate to the different grades. So we provide as much information as possible from the outset, and that is the difference between ourselves and what I think Ron McLone was saying.
(Ms Tattersall) No, I am not really. I am taking it from the point of the grading boundaries. All of us have a very clear procedure, in terms of the marking of the scripts, the standardisation of examiners, they all have a meeting where they are standardised.
(Ms Tattersall) It is the end of the process. No, all the marking has been done, or should have been done, by that stage; and then there is a group of people whom we call 'awarders', there will be a Chair of Awarders, who is the person who makes recommendations to the Chief Executive, and there are the various chief examiners, or principal examiners, for the different components of the examination, and they will have made recommendations as to where they think the grade boundaries should rest. And the awarders as a whole will get that information, together with the statistical data that I have just mentioned, together with a range of scripts, which cover the various recommendations which have been made; and, within that, the awarders have got to try to find the defining mark between one boundary and another, and it is not easy. And if you actually look at the range of decisions which awarders make, some will believe that a mark of, let us say, 40 is the mark, others will believe a mark of 39, others believe a mark of 41, and so on, and somewhere somebody has to come down and make a judgement on that matter.
(Ms Tattersall) I have to say, that did not happen in my board, so I cannot actually account for what you are actually describing; and it does not happen in that way in my board, the recommendations come through to me, as the accountable officer, and in the vast majority of cases the recommendations stand, I accept them. In the very small number where I say, "I am not quite certain whether all the evidence has been properly taken into account," then the normal procedure in AQA is for that then to be discussed with the Chair of Examiners, and some accommodation of view is arrived at. In some instances, I might have to decide that a different mark, and it is usually one or two marks that we are talking about here, would prevail, and, as I said earlier, some of the decisions which I took, in the very, very small number of cases where I made a decision, the majority of my changes were actually in favour of the candidates, they were actually to drop the mark, not to raise it.
(Ms Tattersall) Obviously, I cannot account for what Ron is saying, in terms of it being different. I think what he was saying was that some of the statistical information, which we introduce at the very outset of the awarding meeting, because we believe that to be transparent, open, above board, everybody knows what is going on, might have been introduced into the OCR procedure at a later stage.
(Ms Tattersall) The Code of Practice very clearly lays down that the script selection is made by the awarding body staff, in connection with the recommendations which have been made by the principal examiners for the unit or the paper concerned; so I doubt very much if there are differences really in how we operate there, because there is a very clear statement in the Code of Practice.
(Ms Tattersall) I think that really is precisely the issue that, as awarding bodies, we took up with QCA in March, when some language, which might have suggested that we ought to be having the same percentage of candidates, was being used, and we took up that issue very firmly and very clearly in the letter that we sent on 22 March, and which then, in my judgement, was resolved by the letter which we had back from William Stubbs. In terms of the quantum shift up, as it were, you referred earlier to a 4 per cent rise, and I think you were suggesting that perhaps it ought to have been a 9 per cent rise, or whatever; now, if you actually look at individual subjects, you will find that there are 8 per cent here, 9 per cent there. Four per cent is the general, overall, national shift across the three awarding bodies; look at it in individual subjects and you will find some very different patterns emerging. And we have not done this analysis yet in AQA, but I am suggesting to my colleagues that we do it, as to whether those shifts were different in those subjects which were modular beforehand from those subjects which moved to a modular system in 2000; and that is an analysis which certainly we can have a look at, and we will be happy to provide the Committee with information later on.
(Ms Tattersall) It has been at round about the1 per cent, sometimes less than 1 per cent, level, but it has been a very marginal change over the years. But, again, if I can pick up on the modular theme, if you go back to round about 1993, when many of the subjects, particularly in the sciences, started to, as it were, go modular, you did actually find the shift then at the Grade E and above level, which was greater than the normal pattern in other exams; and that was in a system where you had a greater facility for retaking than now, because there was a limit on the retakes.
Chairman: I have asked Dr McLone to come back, and he has very kindly agreed, because I think Meg Munn was not happy that she quite fully understood the difference between the two approaches of the two boards; so would you like to rephrase your question, Meg Munn?
(Dr McLone) I do understand what Kathleen is saying, because we have had these discussions many times. They are both within the Code of Practice; the whole thing about the Code of Practice says that you have got a balance between examiner judgement and statistical evidence. I have to say, I am going back a few years now, back to the Midland Examining Group, which was part of one of the first GCSE groups, along with the NEAB and SEG, and so on; we always took a view then that what we wanted to do was to make sure that the examiners had the first go and talked about it and then looked at what the impacts were. It is sort of very much a bottom-up process; in a sense, I think the Midland Examining Group said it was an accountable process, because you could see what was happening with the statistics, because then it was evident. It is true, there are other ways of doing it, and one of the other ways is, as Kathleen has said, to produce a good deal more of the statistics to inform where the scripts are selected in the first place. That, essentially, is where we are at.
Ms Munn: Thank you. I understand.
(Dr McLone) Indeed; and, without knowing the subject, of course, I cannot actually directly comment on what an individual would say. The difference this year has been, it is the first time in 50 years to have such a fundamental change of A level; it is not different in the practice, and it may well have been that, in his subject, or her subject, I would not like to say whether it is his or her, nothing has happened significantly, but this year, in a few subjects, I have to say, in most of the subjects, did not get such substantial issues that have arisen, but in some subjects, obviously in this subject, there was this difference which has come by looking at what they have suggested against statistical evidence that has been more dramatic than in the past. And we have said, and I say again, it is a major shift this year. The way we do it obviously works very well in circumstances when it is maintained year on year and it is a regular, consistent standard, but if you are working, again, with an A2 standard, which, I still submit, we did not know, we had no exemplars, that has provided the issues for some of our examiners, it is absolutely right. Remember, I look across all subjects, he is looking at his own subject.
(Dr McLone) I think it is unfortunate that we had no trialling done before we made such a major change.
(Ms Tattersall) We are not a norm reference system, I think that is the first thing that I would wish to say; we moved away from norm referencing many, many years ago. I think, at some point in the 1980s, A level ceased to be a norm referenced examination. Nor are we fully a criterion reference examination, but we have, as it were, moved along the scale more to that by defining some criteria to underpin the grades, and those criteria are defined at A level overall. If we moved entirely to a criterion reference exam, then I think you have got to take the consequences of that; namely, if you have not mastered whatever is determined for the grade, you will not get that grade, however, as it were, good you are.
(Ms Tattersall) But what we do have is a system which is a soft criterion referencing, for want of a better term, where there is some compensation for a weakness in one skill area, with strength in another, and, in that sense, you could say, it is a little bit of a fudge, when it comes to the criteria. But it is a system which does reward attainment at the more general level than some very specific criteria would do, and I think it is a system which has served students exceedingly well over the years, and, indeed, if you look at GCSE it is exactly the same sort of system, if you look at Key Stages 2 and 3 then I think what we are talking about there is a pre-determined level of attainment, which is only slightly moderated when the students have actually done their SATs. Some of us would say that, in some ways, criterion referencing is fine, but it is when the students actually do the exam that some of the criteria begin to break down, because it is not like that in the real world. So, in a sense, I would argue very strongly for the soft criterion referencing system that we have, provided we have a little bit more definition of those criteria, but not so specific that we are going to cut people out of the grades.
(Ms Tattersall) I think we would have exactly the same issue, but we would have to have descriptors which enabled teachers to mark work consistently; and the fact is that students do not perform in consistent ways, and, therefore, there has to be, as it were, some sort of compensation for the way in which students strive to meet the criteria, and that is what our system does. And I think it is irrelevant, whether it is internal or external assessment, to actually apply the criteria that we have. I am all for making the criteria more explicit, trying to reach criteria which are better understood by everyone, but I really do think that we would be in trouble if we tried to rely entirely on criteria for our system.
Chairman: I think that we have to end the session now. Thank you, all of our witnesses today, who have taken the time also to enjoy a rather different format we have played with today, and thank you very much for being so flexible. Can I say, to quote John Kerr, perhaps a mission to explain, I was thinking this when you were talking about, of course, everyone knows we have moved from norm referencing to soft criterion referencing, but there are a few people in my constituency who did not quite realise that that had occurred. Perhaps it is part of the role of the QCA and the examining boards to tell parents and students that that is the case. Thank you.
KEN BOSTON, Chief Executive, BEVERLEY EVANS, Deputy Chief Executive, and KEITH WELLER, Head of Qualifications Division, QCA, examined.
(Dr Boston) Well, Mr Chairman, I will start by saying that I certainly have no magic wand, and I am not at all sure that the path into the examinations in January and in June will be smooth; there are some major problems and some major risks ahead of us. Certainly, we will be able to respond to the Tomlinson recommendations by the end of November; we will have before us then better generic statements of standard, we will have a revised Code of Practice, we will have made considerable progress in getting specific exemplars from this year's exams, for A2, which we did not have before, but getting that all understood in time for the 2003 examinations is a big challenge. We also have some enormous logistic problems ahead of us, in running the examinations. We have been stretched in the past, or the awarding bodies have been stretched, to find sufficient examiners; we have this incredible process where 24 million scripts go round the country in a matter of weeks. We have little control at the moment, or virtually no control, but need some control, I believe, on the number of late entries for examinations; and I am not at all sure, unless we pull all this together into a better managed system, that we will not have strife ahead of us again. Now so far as the QCA is concerned, I think it needs to take a far more directive and management role, so far as its powers allow, in determining what goes on. I am not at all sure, for example, that there is real benefit in having awarding bodies able to take slightly different, but nevertheless significant, approaches to implementation of the Code of Practice. I am not at all sure, at the moment, until we have done further work, that we might not run into trouble with one awarding body, or several, again. All of these things urgently need to be attended to. Now so far as the QCA is concerned, as the independent regulator, it needs to have the authority and the credibility to be able to make statements publicly about the state of the examination system, be believed and have the power to fix them; it needs, in my view, to have some degree of greater distance from Government. I do not believe that there has been any evidence of Government interference in standards, or in the work of the QCA, or in the work of awarding bodies, but if it is to be a credible public authority there needs to be the appearance of independence. The other side of that is, there needs to be greater distance, I think, too, between Government and the awarding bodies. If I were here to regulate a financial market, a market in financial services, I would expect the providers of financial services to be totally at arm's length from Government, and for the regulator to bridge the distance between them. Similarly, there must be, in my view, conspicuously, clear blue water between the awarding bodies and Government, and the bridge across that is the regulator; now that is not conspicuously apparent at the moment. I believe the QCA has acted independently, from my reading of all the documentation, and, believe me, my mind has been concentrated wonderfully on the documentation over the past few weeks. I do not believe there is evidence of political interference or pressure on QCA. I see no evidence, but I take Mr Tomlinson's report, of pressure from the QCA on the awarding bodies. But it is clear that the independence of the organisation is not transparently there, it is not unambiguously accepted, and it needs to be, in a far stronger and clearer way.
(Dr Boston) I think Ofsted is a very good model in the education area, yes. There are other models of regulators, I guess, both here and overseas, in completely different sectors, in my home country, for example, in the transport sector, the financial services sector; there is not in the education sector because education, in the states (where I grew up ?), in Australia, is not run on the basis of a competitive market between organisations, competing on the basis of not price but quality of service.
(Dr Boston) There seems to be, from the evidence I see, quite close contact between DfES (the Department for Education and Skills) officers and individual officers in awarding bodies, at a variety of levels and for a variety of purposes, all of which, I am sure, are benign. But, nevertheless, in a situation where there is a regulator, I believe that relationship is not a desirable one.
(Dr Boston) The reason I became interested in the job was because the QCA does have a very high international profile, it is the international benchmark, as a qualifications and curriculum authority, far broader than simply something concerned with A level examinations. There are, in fact, 117 different awarding bodies, many of them, the majority of them, by far, in the vocational area. Qualifications and curriculum authorities have been introduced in many western countries fundamentally for the purpose of maintaining standards, enhancing standards and responding to the workforce skills needs of the countries in which they operate, the building of social and human capital; that is the job that I came to do. I also came to manage the operation. Now I have walked into the situation where I find, since taking up the position, which I took up on 12 September, but, being here in August, just privately, watching the examination system and the results come out, that a whole new set of priorities have emerged, as a result of this real problem that has occurred with the A level examinations, and which is the product of a series of mistakes made by Government, by QCA, by awarding bodies, and a lack of common understanding across the country about what standards are and how they are determined, and we have seen that lack of understanding in here today. Here is a real problem that needs to be addressed; and that is my task, to take that on. I am simply saying, there is no magic solution here, it is a long, hard row ahead of us, and I can give no guarantees, except the capacity to point the organisation in the right direction, work with the awarding bodies and Government and the Head Teachers Association and teacher associations to try to get it right.
(Dr Boston) The job is that of a regulator, I report to a board, the board is appointed by the Secretary of State, but it would seem to me that the QCA is an organisation outside the Civil Service per se, it is a non-departmental government body, it is there to maintain and defend and protect standards, it is there to guard standards, it is the watchdog, and the watchdog occasionally must bark.
(Dr Boston) I was not here when the events that were discussed by Mr Tomlinson, and have been referred to in the discussion today, were alleged to have taken place; all I can do is read the documentation available to me, and I find no evidence there, I read Mr Tomlinson's report and I take that at face value. It is not for me to say, in that instance, when I was 12,000 kilometres away, whether undue pressure was put on them or not.
(Dr Boston) That was not what I said, and I do not have a view on that matter.
(Dr Boston) I would guess the answer would be, yes, in that all our attention has been focused on dealing with this issue, of which we are, along with others, partly the cause. So that the key priority at the moment, the real key priorities are to overcome the problems in the examination system, urgently. There are two levels with that, implementing Tomlinson, the first part of his inquiry, and attending to the immense risks that reside in the logistics of the operation, and restoring the credibility and authority of the organisation; they are the two key priorities. And the international work takes, clearly, second priority, second preference, to those.
(Dr Boston) I think that the key mistake was not perhaps being vocal enough with the time-line issue, when one goes back to look at the way in which this was implemented. A change of this magnitude really should take three years of development and two years of piloting, and the piloting is so critical, to get the exemplar papers, which set the standard. We have a generic statement about standards, about the A2 being harder and the AS being an easier paper, as it were, we have grade descriptions for English, for History, for Physics, for Chemistry, for Mathematics, which describe in terms of perhaps 200 or 300 words quite analytical, well put together documentation of the sorts of knowledge, understanding and skills and analytical abilities a student should have at each level of a grade description; but you cannot take that any further and operationalise it until you have actually got exemplar material. Now that is what we did not have; and, in fact, the time-line was very rushed. The final specifications were finalised in January 2000, teaching began in September 2000 and the first examination was in June 2001. Now it is quite clear that that sort of time-line makes it very difficult to introduce a change of this type without real risk occurring. Reading through the documentation, it drew to the attention of Government several times the impact of the time-line, so did the awarding bodies, but perhaps we should have been more strident at that time about where we were heading. I think that was the key problem. At the same time, of course, as all this was happening, a whole lot of other things were going on; there was Key Stage testing going on, there were changes to the GCSE, to the GNVQ, there were examiners being stretched to the limit in all sorts of things other than A levels, specifications, or syllabuses, were being written everywhere, 3,500 new specifications were written between 1998 and 2002, when you go to the exact dates, it works out at an average of 15 new specifications, new syllabuses, a week; now that is stretching an examination system to its absolute limits. And it is quite clear that we have reaped the fruit of it in this most recent event; and there is no sort of quick fix, that it is only a matter of attending to that, or attending to that to get it right.
(Dr Boston) I do not have any specific criticism of the awarding bodies; one can point to areas where, with the benefit of hindsight, there could have been better performance. The problem really only lies with one awarding body, and there are two issues that seem to me to reside there; one is the different approach to implementing the Code of Practice, which seems minor, and might be minor, but if it did have an impact, and I do not know whether it did, but we would need to sort that out, then that impact clearly had a very bad effect upon a generation of young people. The other is this issue of standards, and the definition; with respect, I do not agree with my colleague, Dr McLone, that we are really talking about a new standard, with A2. In my view, it is better to look at there being two examinations, of different levels of difficulty or demand. For 50 years, the A level has been judged by one examination at the end of two years; we have now changed to a system where there is a less demanding examination at the end of the first year, or, if you like, in the language of the students, an easier examination, and at the end of the second year there is a harder examination. I do not believe it is an examination pitched halfway between the old A level and the end of first year university standard, that is inappropriate for these young people.
(Dr Boston) I think that we would all be more certain, and I agree with Dr McLone and others, if we had the exemplar material that would actually show you. I have been back through, say, some History papers, I looked at the A2 History papers and the AS History papers, and I looked earlier today at a question on Nazi Germany. Now the AS paper was a question that took students to some source material, one was a cartoon and some short pieces of text, and then had a 30-mark question which was broken up into sections of five marks, five marks, ten marks, five marks, and something else. The A2 paper is a paper which also had source material but it was heavy-duty, political literature, and then the question was only in two parts, a ten-mark question and a 20-mark question, making the 30 part, and was far more demanding. Now, the young people who do that exam, that is a hard exam, and the other one was an easier exam; but the two of them, when they come together, and you will perform probably not as well on the harder exam as on the other, when the two of them come together, that gives you the A level, and it is our job to ensure that the awarding bodies, that the standard of that is the same now as it has been for 50 years. Now we will only do that with certainty when we have got the exemplar material, and we did not have the exemplar material for this round of exams; but my starting-point is, let us talk about level of demand rather than a standard, there is one standard, and that is the A level standard.
(Dr Boston) I think the key to that lies in the revision of the Code of Practice, that Mr Tomlinson has asked, or directed, that we undertake, which should remove the capacity for different approaches in that way.
(Dr Boston) I would seek the advice of the experts in my organisation, and I am not in a position tonight to say which is the better approach. But, clearly, in the revision of the Code of Practice, this is an issue we need to address, and we are addressing that through the A Level Programmes Board, which meets tomorrow again, its second meeting since Tomlinson came down, it includes the heads of the regulatory bodies of England, Northern Ireland and Wales, the heads of five awarding bodies, representatives of the head teacher associations, and the teacher associations, and it is in that body we are going to sort it out.
(Dr Boston) I might ask my colleague, Mr Weller, who is, in fact, doing the detail of this, to come in, Mr Chair, if you would be agreeable.
(Mr Weller) The Code of Practice is revised every year, actually, for A levels, and for other examinations, and it is done in the light of what happens operationally; we look at the Code in action, we look at it with the awarding bodies, and we adjust it, if it is not doing the job effectively. There are always areas, I think, that the Code cannot go into in the finest level of administrative detail, we would be doing the awarding bodies' job then, and there are always areas where you decide there are various ways of doing the job perfectly reasonably, and that is the way the Code operates. If it transpires that, in doing the job reasonably, through various methods, we have some cause to doubt whether they are equally efficacious, we will talk with the awarding bodies about that and establish which of those methods should be codified. We do seem to have an issue here where there might be some of that kind of discussion necessary.
(Mr Weller) Both.
(Ms Evans) I think, when you spoke to the QCA in May, it would be fair to say that the problems that we thought were going to create most difficulty in the A level system this year were ones to do with the delivery of the exams, the physical delivery, making it happen, getting papers into schools and colleges.
(Ms Evans) Certainly, there were specific problems with Edexcel, but we did not just concentrate on Edexcel, we were looking at the issues right across the range of awarding bodies. I think it was referred to earlier, examiner supply, for example, was an issue that we had to pay very great attention to, and, indeed, one of the awarding bodies, in the end, was not able to have all its papers marked to the right timescale; that was a matter of great concern to us, we were managing it very directly, right up until the results were announced, in fact. So I think it is fair to say that our attention was focused on those issues, because we had identified those as being the highest risk issues to the development of the system.
(Ms Evans) As with Ken Boston, I have not been here for the whole of the period when QCA has been working on the new exam system.
(Ms Evans) I have been here two years, and what I was going to say is, looking back at the record to 1998, what I see is an enormous amount of exchange, dialogue, correspondence between QCA and the awarding bodies and the teacher associations, and head teacher associations, in examining the very issues that you are describing, what the standard should be, what the A level standard will be in future, was it going to be a different standard, or was it going to be the same standard, and how the AS and the A2 components of those fitted in. So that dialogue had been taking place since 1998, and, in fact, I think, since I have been in the QCA, since the year 2000, that dialogue had seemed almost to have come to an end by the time I joined QCA. There is a record in the early part of the teaching of the first A levels, in the autumn of 2000, and that, to me, would appear to be the tail-end of that debate and that discussion; it was not an issue that was continuing to be discussed between ourselves and the awarding bodies during the last two years, the main issues for us were delivery.
(Ms Evans) Perhaps I could correct what I said earlier, because that was not what I meant. Certainly, the organisation has a whole division of staff who attend awarding body meetings to observe what happens when decisions are taken about awarding, and their role there is to determine that our Code of Practice is being followed correctly; that was going on throughout the whole of the period. I am sorry if I gave you the impression that that was not happening.
(Ms Evans) Dr McLone was in a lot of correspondence with the QCA since 1998 on what he believed were aspects of the process that he wanted to clarify; but I think it is fair to say that, certainly during 2001 and 2002, there was very little exchange that went on, the awarding bodies were not saying to QCA, during 2002, "We're very unhappy and very uncomfortable about how to do awarding this year," that was not being said at all.
(Mr Weller) Chairman, may I comment. I have to confess to having been there longer than Beverley. Over the period from May '98, when the decision was made to proceed with the new curriculum, until about May of this year, we had some 35 meetings of a technical kind with the awarding bodies, and they went right up to April or May this year, and they were concerned with all the technicalities of the examinations. I think it would be true to say that the issues with grading and awarding were resolved, we believed, with the awarding bodies ahead of time. I think we were all conscious that we would have liked more exemplar material, that point has been made a number of times, but I do not think we were leaving the issue as though it had been finally resolved early at all, we went on meeting, indeed, up until April this year, on those issues.
(Mr Weller) We worried about it a good deal, and we had a very extensive discussion on it at the time that the first advice went to ministers at the time on the reforms; and, indeed, that advice was public, as was the Minister's response, that is all public. We talked long and hard, in a whole host of consultation conferences and through written consultation, with all the users, all the stakeholders, including the schools and colleges, about the options of a 40/60 or a 50/50 balance. It has to be said that, while two-thirds of those responding were supporting the introduction of an AS examination, it was a very popular reform, still is, I believe, a popular reform, the one thing that they really worried about was the 40/60 split proposed by Ron Dearing, they worried about it in terms of what it would mean in relation to perceived value, by universities, by parents, they worried about the complexity, that 40/60 would have to sit alongside 50/50 in the university tariff system, 50/50 performance tables, and the main message from that consultation was, this is a bridge too far, in terms of complexity. So the decision was that we make it a 50/50 balance, we have an easier part, or a first-year part, and we have a second-year part, and the combination of that first-year and second-year part, more and less demanding, makes the A level standard. Once the decision was made, it was important for us to get it right and work at it with the awarding bodies, and that was what we did, over those 35 meetings.
(Dr Boston) I think the comment has a good deal of substance to it, and, in fact, it relates to the quinquennial review which took place earlier this year and made some similar points. We are looking at that very closely, and are about to respond to Government on how to implement the quinquennial review, or to deal with it in another way. One of the key issues was that the regulator should not also be the provider of examinations, and with our statutory tests we are the provider of the examinations; the suggestion, or the recommendation, was made that we look closely at separating that and at moving it away. Now we either grow some capacity outside the organisation to take on the statutory tests, we hand it to an awarding body, and I do not think there would be necessarily immense public confidence in that, at the moment, or we contract it out in some other way, by tender, or we live with the ambiguity and continue to run the Key Stage tests ourselves, until such time as they can be moved elsewhere. The public has confidence in those tests, and they are running smoothly; it is not a problem that needs to be fixed, it is a philosophical point that a regulator should not also be a provider, and I agree with that, but, nevertheless, there must be an element of pragmatism in the implementation. I have no doubt that the QCA will change and develop over a period of time, particularly if it has a different reporting relationship, in due course. None of us is in the business of defending the status quo, in fact, in my view, the status quo is the only option that is not on the table; another way has to be found.
(Dr Boston) I think all of us failed, in the key strategic thing we all have to achieve, and, by that, I mean Government, the QCA, the awarding bodies, teacher associations, and everything else, we failed to communicate the nature of the new standard, and we are still grappling with it tonight, and that is the key issue that we have to resolve. The Programme Board was set up to do that, it is working very hard, we will meet the Tomlinson deadlines, we will get clarification, but the promulgation of that and growing it into a community understanding of what it is about is going to be a very complex process, and I doubt that we are going to have that delivered into the national psyche by the time of the January examinations.
(Dr Boston) I think the QCA should be taking the lead in it, and that is what we intend to do; but, of course, the training and development, the in-service education training programmes for teachers, are delivered by the awarding bodies. But we need to be far more rigorous, I believe, in our scrutiny of that, just as we have been rigorous in our scrutiny of the actual awarding process. We need to have a far greater eye to the support that goes into working with teachers and getting a general community understanding; and there are some things we must do ourselves, as the regulatory body.
(Dr Boston) I would not put it that way. I would put it as the QCA properly carrying out the role for which it was established, and that role, from time to time, will involve saying to Government, "This is not a possibility; if you proceed to go down that track, or on that time-line, you're going to run into trouble." It seems to me that that is what the nature of a regulator is, and that, unless a regulator is prepared to do that, credibility and authority will never be established; you cannot legislate for credibility and authority, you have actually to demonstrate it by performance, by the quality of what you do, and that is where the QCA must position itself, in the public arena.
(Dr Boston) No, I cannot say that. The QCA did everything it could to make a clear statement about how the A2 standard and the AS standard related, and, indeed, I understand it has been on our website since 2000; we did everything we could to write rich, meaningful statements about grades in all the subjects, which examiners and teachers could understand. What we did not do was deliver on exemplars. You do not really know what a grade B is at A2 level until you have looked at a thousand scripts, from young people across the country, how did they handle that question on Nazi Germany, what was the depth of the analysis, the depth of the synthesis of argument, how did they deal with the synoptic issues, you do not know that until you have got that material in front of you. Now that was not a fault of the QCA or of the awarding bodies, that was a fault of the time-line; we launched into the first A2 examinations in summer without that pilot material behind us, and we should have had two years of it. We were okay with AS, because that has been sat three times, but we were not able to deliver on the A2; even though the A2 had been sat during the trial process, the grading had not been sorted out, and so the exemplars simply could not be used.
(Dr Boston) The QCA should have done more, but it is not in the sense of being able to write down, in specific detail, other than the generic statement, what the standard is in History at A, at B, at C, using real substance, that has to come from the exemplar material. But the fact that we even have a discussion now about AS and A2, and whether there are standards or level of demand and how they relate to each other, indicates that the QCA, along with other agencies, has not delivered adequately in implanting that understanding in the minds of the profession and the community.
(Dr Boston) Mike Tomlinson has produced a report which has looked at those issues, and I really do not want to add anything to what he has said, because he has looked at a wider range of evidence than I have, I have simply looked at what has happened in the QCA.
(Dr Boston) I would not want to comment one way or the other in relation to individuals.
(Dr Boston) I did not say that. I have not discussed the structure of the QCA. The QCA is an organisation which, as I have said publicly, now has to reinvent itself, it has to establish very clearly what its priorities are, it has to be very clear about what its strategy is, it needs to align its structure to deliver that strategy. There will be changes in the QCA, but I do not believe that the problem which has emerged here is a problem which can be driven home to particular individuals, either in the QCA or elsewhere, and say they were guilty. It is a compound of a series of things that should not have happened, rushed time-lines and other things, which, with the benefit of hindsight, coming in as a new person, I can see, and others are seeing at the same time. I am not so sure, if I had been here, whether I would have seen all of those problems emerging, but, the fact is, they happened.
(Dr Boston) What I am saying is that that is the wrong question to ask, if we are going to make progress with this. The issue is, we are dealing with A levels as they have been for 50 years; the change is, we are arriving at it now from two papers, one of them is a hard paper, one of them is an easy paper, relatively. But that is language which every student who takes the course understands, there are two papers, and you add together the scores on the two of them, one you take a year before the other, and you get a result.
(Dr Boston) Mr Chairman, I am not wishing to comment on that issue, because I have been rather preoccupied with things other than UCAS and university entrance, and I have not thought that fully through; but, clearly, it is part of the work that we have to do in implementing Tomlinson, because, clearly, this is a major purpose to which the result is put.
(Dr Boston) Yes, I do, I do recall that. Mr Chairman, I was made aware, by a senior official of the Department, that discussions were occurring between members of the Department and the awarding bodies on what would happen if Tomlinson sought to recommend regrading, and that was accepted by the Secretary of State. I was concerned about that, as the regulator, and concerned because, earlier that same day, we had been examined by Mr Tomlinson, and we had made it clear, in response to questioning from him, that we ourselves were having no contact with the awarding bodies while his work was in progress. Now I contacted a more senior official at the Department to express concern at that, as the regulator. I have no objection at all to the Secretary of State sorting out the various scenarios, as it were, that might emerge from an inquiry and seeking advice on what to do with each one of them, but, the reality is, the conversation should not have been with the awarding bodies, by the Department, but with the regulator. Because the regulator is thoroughly across, because of its role as a scrutineer and day-to-day regulator, issues of the capacity of the awarding bodies to deliver, and would be able to advise Government on whether or not regrading was possible; indeed, we were, in fact, doing that, as a result of another request. My concern was not that the work was being done, but that it was being done directly with the awarding bodies rather than the regulator.
Chairman: We are going to suspend the session for 15 minutes.
The Committee suspended from 7.02 pm to 7.15 pm for a division in the House
(Dr Boston) Yes. There are three amendments to the Act; the most substantial one is a new section, 26(a), which does give us the power to intervene, to direct, and we do see ourselves using that power, not necessarily always only to correct what might be some mistake or aberration but to manage the system a little better. I referred earlier to, one of the problems in dealing with the examinations is the large number of late entries that can occur, in fact, there are sometimes young people who turn up on the day of the examination, and papers are photocopied and given to them. Edexcel had, over all qualifications, over half a million late entries at the last examination; now, if we got something like that scale with the A levels, even in proportion to it, it just simply becomes unmanageable, the number of markers that you require goes up enormously. Now, if we are to model the system and manage it correctly, one of the things we are looking at is using that new power to determine that there be no late entrance, or no late entrance after a particular date, except for young people who might be in particular categories, awaiting remarks, or something like that. Now we have not committed to that, but that is an example of the sorts of things the new powers could be used for.
(Dr Boston) They might. I think the community would see the fundamental test any regulator has to face, or pass, or, indeed, a Government has to pass, is, well, it is the equivalent of making the trains run on time, make sure the examination system works; and we have seen in this last month an examination system that has been under extraordinary pressure. The priority now, I think, is to make sure that never happens again, and we do that both by addressing the Tomlinson recommendations and, on the other hand, addressing the issue of logistics.
(Dr Boston) I have no expectation of that, at the moment. It will depend very much, I think, on what Mr Tomlinson finds as he addresses his second term of reference, and it might be that legislation flows from that, possibly in relation to the role of the QCA itself.
Chairman: In terms of the Queen's Speech, I think he would have to hurry up with that. Andrew, you were in the process of finishing your questions, I think. I think you were in full flow.
(Dr Boston) I think the answer is, no. Mind you, I came to that conclusion on the basis of experience in another country, where there are ways in which these matters are handled, and I was coming from that background; but, because of my concern about the matter, I did telephone Mike Tomlinson and report it, because of the discussion, or the examination that he had given us earlier in the day, when the issue of contact was raised. I had no thought that it compromised the integrity of this inquiry, and he quite properly came out the next day and made a statement on precisely that point, and that was fine. Nor, as I said earlier, did I have any reason to think that the Secretary of State acted inappropriately; of course, she was sensible, to look at all possible things that could come out of the inquiry and know how she was going to deal with them. It is just the wrong bodies were consulted.
(Dr Boston) My Chairman. I do not understand the assertion, Mr Chairman.
(Dr Boston) The then Chairman, Sir William Stubbs?
(Dr Boston) I did discuss it with Sir William Stubbs, indeed, before calling Mike Tomlinson.
(Dr Boston) No, I was not in accord with any protest against the Secretary of State. I was concerned that the legitimate request of the Secretary of State had been dealt with by approaching the awarding bodies to ask them whether they could handle a regrading, which was still being considered by an independent inquiry in progress. My concern was that the QCA, as the regulator, had not been the body that was consulted. I had no criticism at all of the Secretary of State.
(Dr Boston) I cannot speak for Sir William, Mr Chairman.
(Ms Evans) I was present when the discussions were taking place between Ken Boston and Sir William Stubbs.
(Ms Evans) No. I think, my recollection of Sir William Stubbs's view, as we were discussing it, was that it was inappropriate of the Secretary of State to have discussed, or to have asked two officials to discuss, those matters with the awarding bodies, rather than discuss them with ourselves.
(Ms Evans) He then proceeded to speak to a number of journalists about a number of matters, including the evidence that we gave earlier that day to the Tomlinson inquiry, and it was in the course of those discussions with journalists that that came out.
(Ms Evans) There was not a discussion of that sort that took place, I am afraid.
(Ms Evans) I am a member of the Department, as you have referred to before in this Committee, but for the period that I have been working in QCA then my role is as a member of QCA's staff, and that is the way in which I have acted.
(Ms Evans) The discussion that took place between us was on the appropriateness of DfES officials having those discussions with awarding bodies and not having those discussions with ourselves.
(Ms Evans) I think it was inappropriate that the discussions were had with awarding bodies and not with ourselves.
Chairman: Right; so you are in accord with your new Chief Executive on that.
(Dr Boston) It is the shortage of examiners, and I think that is going to be exacerbated this year by many people not wishing to examine again, or perhaps examine for one board again; the sheer volume of the assessment that occurs across the country. I do believe examinations here are probably the most excessive in the world for young people, and that we could get equally valid measurements of student performance and progress with less examination. The reliance so strongly on external examinations, rather than some component of it, at least, being internally examined. The notion of having internal assessments externally moderated, which the Secondary Heads Association is advancing in the form of chartered examiners, is, in fact, the norm for many examinations in many western countries and produces valid results. John Kerr referred earlier to issues of technology. The technology that we use is very simple, and it was the subject of some comment in the report, maintaining A level standards, that Peter Baker chaired earlier this year. Our scripts are all marked by single markers, no script is marked by two markers; the scripts from centres move by post to a marker's home, usually, we do not use, although we have trialled, as a general rule, marking centres, where markers are brought in to mark under supervision, and one marks questions 5a and 5b, and another marks questions 6a and 6b, and you get consistency that way. Very little application of technology. We are running here a 21st century education system on a huge cottage industry, in the marking process, and it is just going to fail, unless we move to change the way that operates. Now that cannot be done for the summer examinations next year, we do not have that capacity to move that quickly; but that is the longer-term issue, we have got to get the examination system logistically and technically on a much firmer basis.
(Dr Boston) I cannot say that I have a developed strategy for doing that, at this stage; but I think it needs to be put on the agenda for public discussion, backed up with a lot of evidence and with alternatives, and become a subject with which the community as a whole occupies. I have followed very closely the developments in education over the years, and I understand the pressures that have led to this highly intensive testing programme. As a person who is experienced in this field, although in other contexts, but has been in it all my life, I think there are major problems here, and I am actually more concerned about those problems than the A level issues. From the point of view of the A level and the marking and the limited amount of regrading that has gone on, this is not a system in disarray from that point of view, it is a system that has been through a difficult passage but the causes of that can be addressed, and can be addressed probably in the relatively short term. But addressing the bigger issues of the potential for the system actually to break and not be capable of being delivered, they must be urgently on the public agenda; and I think one of my roles, and one of our roles, in the QCA, as an independent regulator, is to lead that public debate.
(Dr Boston) I think we should be looking at all phases. The 7, 11, 14, certainly there are intervals there, and other systems have similar intervals, but we also have an extraordinary number of optional tests, some of which, many of which, QCA, in fact, develops, that are administered to children. There has to be a balance between assessment for formative purposes, for aiding learning, assessment for summative purposes, so that Government, that has made the investment in education, knows whether they are getting the outcomes that they are investing in; but it does not have to be this extraordinarily intense programme, we have got to leave some time for teaching, not testing.
Chairman: Dr Boston, I think that that is a good note on which to end this session, and, certainly, if you are coming out of your corner fighting for those principles, you will get a lot of support from this Committee. And I hope you have enjoyed your first session in front of this Committee, and we look forward to a long and creative partnership. Thank you.