Publications on the internet
UNCORRECTED TRANSCRIPT OF ORAL EVIDENCE
To be published as HC 1671-iv
House of COMMONS
TAKEN BEFORE the
The Administration of Examinations for 15-19 Year Olds
Tuesday 21 February 2012
John Butterworth, Paul Howarth, Jacob Pienaar and Kate Harris
Rod Bristow, Mark Dawe and Andrew Hall
Evidence heard in Public Questions 463 - 578
USE OF THE TRANSCRIPT
This is an uncorrected transcript of evidence taken in public and reported to the House. The transcript has been placed on the internet on the authority of the Committee, and copies have been made available by the Vote Office for the use of Members and others.
Any public use of, or reference to, the contents should make clear that neither witnesses nor Members have had the opportunity to correct the record. The transcript is not yet an approved formal record of these proceedings.
Members who receive this for the purpose of correcting questions addressed by them to witnesses are asked to send corrections to the Committee Assistant.
Prospective witnesses may receive this in preparation for any written or oral evidence they may in due course give to the Committee.
Taken before the Education Committee
on Tuesday 21 February 2012
Mr Graham Stuart (Chair)
Examination of Witnesses
Witnesses: John Butterworth, Educational Writer and Chief Examiner, Society of Authors, Paul Howarth, UK and International Managing Director, Nelson Thornes, Jacob Pienaar, Managing Director of Schools and Colleges, Pearson UK, and Kate Harris, Managing Director, Education and Children’s Division, Oxford University Press, gave evidence.
Q463 Pat Glass: Good morning everybody, and thank you for coming. Graham Stuart, the Chair of the Select Committee, is going to be a little late this morning. I am stepping in until he arrives, but when he arrives I will move over. As you know, we are holding an inquiry into the examinations system in England and Wales at the moment, and one of the things that the Select Committee wanted to get out of today’s session, if we can, are recommendations, as part of our report for how things can improve. When you are giving your evidence, if you can remember to give us what you think would be snappy answers and recommendations.
I want to start off today by asking if things have become just a little bit too cosy in the examination system, with examiners writing textbooks and, in some cases, question papers, and textbooks endorsed by exam boards. Also, why do endorsements matter? I will kick off with that. Do you want to start, Kate?
Kate Harris: Thank you. First of all, thank you for asking me to come to the hearing today. The short answer to that question is probably yes.
Pat Glass: Could you speak up?
Kate Harris: Sorry, okay. The slightly longer answer is to say that where the same group of people are setting and marking examinations as well as publishing the resources for them, there are a number of risks around that for children’s education. Those risks fall into two categories. The first risk is that having quasiofficial textbooks limits choice, because it takes a brave teacher not to go for the official textbook. Those relationships can be either structural relationships about a publisher and awarding body being in the same ownership, or they can be about exclusive endorsement. What that creates is an official textbook, and as not all teaching and learning situations are the same-you have teachers with different sorts of experience and confidence levels, and you have children with different learning styles-choice being skewed by those relationships is not in the interest of children’s education.
The other risk in those relationships-those exclusive partnerships or structural arrangements, where the people who set and mark the exams are the same group of people who create the textbooks-is that they can undermine trust in the examination system, because the implication is given that with this official textbook, it is in some way giving the answers, and the examination does not, therefore, have the same validity and integrity as it might otherwise have had, which is not in the interests of children.
Paul Howarth: From a competition perspective, the UK market has an awful lot of publishers operating in it, so there is a lot of choice, which is ultimately good for teachers. Based on our experience, people make purchasing decisions for lots of different reasons, whether it is cost, quality or quantity. Therefore there are challenges in identifying examiners and using them as authors, the challenge for a publisher being that often the awarding body and the publisher want someone who really has in-depth knowledge of their particular subject. That is often limited; there is a not a great deal of choice of people out there who make a good author and also understand the syllabus being published.
In a perfect world, what we would have is someone as an examiner who could identify and interpret the syllabus and a separate author. That is not always the case, but we try to identify that where we can. We worked very closely with AQA-when we had an exclusive arrangement-to make sure that there was an absolute separation between ultimately what happened from the exam perspective and what happened within a textbook.
Jacob Pienaar: I personally believe that what is extremely important for education outcomes is proper alignment and coherence between various aspects of education. In other words, what your curriculum says, what happens in the classroom, what will be in your teaching and learning resources, and what is ultimately assessed should be coherent; it should pull in the same direction. We believe that is the right thing to do. Therefore, we would not agree that necessarily these relationships have become too cosy.
However, as Paul has indicated, awarding bodies do two things: they will develop specifications, but they will also write exams, which they eventually award. We do believe that as much as we need coherence and alignment, you need to be careful when it comes to examinations. We would agree with that-the stakes and the risks are too high in terms of public confidence.
In terms of recommendations, what we should do is make absolutely sure that the examinations do not get compromised, while at the same time not fragment what we do to such an extent that you lose the coherence and alignment that I was referring to earlier. I also agree with my colleague that choice is absolutely important. I was a teacher myself for many years, and I know as a teacher I did not want anyone to tell me exactly what I should be using. But I also know I had powers of discrimination as a teacher; I knew that I could exercise some choice. We have loads of evidence in this market that teachers do exercise their choice, and that they are not necessarily simply going one way because someone tells them to.
Q464 Pat Glass: The Society of Authors has been quite critical of this system.
John Butterworth: First of all, I am a member of the Society of Authors, but I do not have a script from the Society of Authors. They have asked me to relate my experiences; I like to think maybe I come into the category that Paul said was a "perfect" author, because I am both an examiner and an author.
However, I came to be an examiner by being an author first. I do not think it is true to say that all examiners of a syllabus are necessarily going to be the best authors for a textbook, except in the sense that they know the specifications inside out; it does not mean that they know the subject inside out, and it does not mean that they can write particularly well. From my experience, I know I have been given a commission to write a book without any real quality control first. I have also written books that I have submitted and had sampled in the proper way, and judged in the proper way.
But in the case of one publication that comes into the category of being a very specific textbook for a syllabus, I was chosen more or less automatically, because I was the Chief Examiner. I wrote it with a colleague, a coauthor, who was a Principal Examiner. That is why we were chosen to do it. There may well have been authors out there who could have done a better job. Moreover, I do know that one publisher was interested, one with a very strong reputation in the subject I write and examine in. They were thinking of writing a book on the subject. They pulled out because they knew that there was going to be an endorsed book with a partner of the exam board I worked for. I have very severe misgivings. This is not sour grapes, because I am ostensibly the one who has gained from this partnership, but I do not think it is the best practice.
Q465 Pat Glass: That is the evidence we have been receiving in this exam board. The two real concerns are firstly, that the author is writing to the specification, the specification is delivering the examination, and the teaching is to the exam; therefore, there is a vicious decreasing circle that is limiting the curriculum. Secondly, the relationship is cosy and is also limiting the curriculum, because unless you are writing the endorsed book, it is not going to be picked up by teachers. Is this conflict of interest perceived in the system real or not? Who wants to go first? Kate? Quick answers would be good.
Kate Harris: I think that conflict of interest is real, and it comes not just from the examiner-the situation John was describing-but also from the way that companies present and brand their materials. I could, for example, show you a specification for an examination board, and then the book that is published for that specification, which is branded in exactly the same way: not just the same logo but the same cover design. Would it be helpful to show that quickly?
Pat Glass: Yes.
Kate Harris: That is the specification; that is the book that supports the specification. It takes a very brave teacher to say, "I am not going to use that book." Jacob said he has lots of evidence of teachers being independently minded, and of course many teachers are independently minded. But we also have evidence of going in to schools to sell materials for particular specifications and teachers saying, "No, we are buying the official book." When the official book is that clearly branded, I would say that the conflict of interest is real.
Jacob Pienaar: Once again, our view would be that obviously one should always look out for potential conflict of interest. What we all should be doing is managing it as appropriately and strongly as we possibly can, but there is a risk that it becomes a bit anecdotal, which might not be fair on teachers and might not be fair on examiners. There is a very wide variety of activities.
Another example would be where Pearson also publishes for other awarding bodies. We would publish for an awarding body like AQA, for example, that until very recently at least had an exclusive endorsement partnership. Two years ago we published in a major GCSE subject; in other words, we put an unendorsed text into the market, and at least half of the centres AQA sent us selected that particular textbook. We should be careful not to generalise here and imply something about teachers that might not be true.
Q466 Neil Carmichael: Let us try to put this into perspective. Some evidence at the Committee has suggested that there is too much consistency between a textbook and exam situations. To what extent do you agree with that evidence? Can you put some scale to the problem?
John Butterworth: I am sorry, did you say too much?
Neil Carmichael: Some evidence to this Committee has already suggested that there really is too much similarity between a textbook and exam outcomes, and this is limiting the opportunity for children to explore intellectual development and so forth. How do you feel that should be measured? Is it a big problem or not such a big problem?
John Butterworth: It is a big problem; I have come to feel more and more that it is a big problem and a growing problem. Some people have used the term "backwash"-I have only quite recently come across this expression, but it is in several of the written submissions-to refer to the effect that the exam is having on the way classroom practice takes place, in order to meet the exact requirements of the exam. I have come to the view that things are the wrong way round. There are very good books on subjects prior to the specifications being written.
In a better system we would have our awards based on the good books that are out there on the subject itself, on the broad subjects, meeting those as the syllabus, rather than an exam creating its subject, in effect, or creating a subject within a subject that is then exclusively matched by a textbook, which in turn becomes the book on that subject. I take it that is what this word backwash means, and it is a good analogy for what is happening.
To my mind the perfect model is the school library, where you have a selection of books on a subject. It is a long time since I was a sixth former, but I remember the excitement of going from being taught in a classroom to going to the library to study, and being able to browse and look at different books on the subject I was studying. To my mind, if you have a set of class books that is the only one the school can afford, or the only one they feel inclined to buy, that is all going to go. I do very much back the Society of Authors campaign to have the emphasis on retaining libraries in schools; that is where you have the centre of diversity in the material you have to place in front of the students.
Q467 Neil Carmichael: Jacob, would you like to come in?
Jacob Pienaar: Back in the ’80s I was the textbook writer of a series, which was probably the most idealistic thing I have ever done in my life: it was meant to change the world. I fundamentally believe that we as publishers have that role. We have an absolute obligation to have depth and breadth in what we do. If it does not happen on occasion, it is certainly our responsibility to do something about it, because I think it is the right thing to do.
We have to remember, though, that what you put in your learning resources does not guarantee that is the way it will be taught. If at the end of the day a teacher does feel pressurised by maybe the system of accountability, or they are really nervous about the attainment of their learners, it might be that they choose to teach in a narrow way, and that would be sad and wrong. That is where our role is to try to enrich, which means to help teachers ultimately go beyond that narrow "teaching to the test" thing, which we are all worried about and we are all critical of when it happens.
Q468 Neil Carmichael: So you hope teachers will push out the boundaries, despite-
Jacob Pienaar: Absolutely, but when you look at the breadth of publishing in our industry, it is fair to say that there is some absolutely fantastic publishing, but if the temptation is there to narrow anything, whether it is in teaching or the production of learning resources, we should guard against that heavily, and we do have a responsibility in that as well.
Paul Howarth: We recognise that teachers very much teach to attain a position on the league table; they want to pass exams and make sure that their school goes up-or at least maintains if not goes up-on the league table. The pressure, and how we worked with AQA when we worked as their publisher, was to ensure that the text was written in such a way, and the ensuing exam questions were separate from the textbook, that there was not a link there. The rationale behind the exclusive endorsement that we had was to ensure that the smaller subjects were published for, which would not be profitable from a publishers’ perspective, and would probably drop off a school’s curricula because there just would not be this selection available. From our perspective, that is why exclusivity worked at that time.
The market has matured and moved on and away from that now, but we need to be very well aware that what is written in a core textbook does not ultimately go to answer the exam questions at the end. What is very nice is the fact that teacher communities are set up and there is a lot of freetoair resource, which gives the extracurricular learning that was perhaps missing in some of the core texts.
Kate Harris: I agree with a lot of what has been said. The key thing one wants is a rich and diverse set of resources for teachers to choose from, so that they can choose the best resources for the particular children they are teaching. Some of the current relationships between awarding bodies and publishers do not mitigate for that because it does tend to have this quasiofficial textbook.
To perhaps put it another way, there is competition at the examination level-that is absolutely true-and then there is competition at the publisher level, where there are a number of people. Where that gets skewed is where the relationship between the awarding body and the publisher implies that there is an official textbook. As a parent, I want my children to have the best books for them in school; I do not want their teacher’s choice being skewed by exclusive arrangements, official books, or even semiofficial books.
Q469 Pat Glass: If this Committee were to recommend that the relationship between the exam boards and the publishers should be separate and that there should not be endorsed books, you would see that as a good thing?
Kate Harris: What we would like to see is a guarantee of actual and perceived independence between awarding bodies and publishers. At one end of the spectrum, that might look like structural separation, although it would not necessarily deal with these exclusive partnerships. At the other end of the spectrum, it could be voluntary regulation, but the JCQ, the Joint Council for Qualifications, had a voluntary code of practice around this, and it manifestly has not worked.
Paul was saying that when they were working with Nelson Thornes that there was not that implicit or even explicit "these are the books that you must use", but the previous Director General of AQA is quoted on the back cover of the Nelson Thornes AQA arrangement: "Using Nelson Thornes support materials means teachers and students can be absolutely confident that they are learning what we want them to learn and following the course as our examiners intended." That is credited to the Director General of the awarding body. The voluntary code of practice has been pushed to breaking point. What we want to see is guaranteed actual or perceived independence between those setting and marking exams and defining specifications, and those who are publishing resources for them.
Q470 Neil Carmichael: With that answer, you presumably would not agree with Tim Oates in his assertion that a textbook should really be part of the assessment process, if you like-very tightly bound into it.
Kate Harris: I do not agree with Tim on that. I believe that plurality and choice are really important principles, and independence between publishers and awarding bodies is another really important principle, because not all teachers and learners are the same.
If I could give an example, I was speaking with the head teacher of a very selective boys grammar school-a state school. He was saying that he was doing a particular specification for Alevel, and that Alevel had, if you like, official endorsed partnership textbooks published alongside it through an exclusive arrangement with a publisher. He and his head of science chose for those boys books that were much deeper and richer, went much further, went way beyond the syllabus, because the young people he was teaching should be aiming for the A, A* grades. He had complaints from parents saying, "My child is being disadvantaged because you are not using the official book."
I know that is deeply anecdotal, but it goes to illustrate that not all teaching and learning situations are the same. What I want is for there to be choice for teachers in choosing the best resources for the education of the children they are teaching.
Paul Howarth: But in that example there is choice, and he made the choice not to go with the endorsed text.
Kate Harris: He did, but he was under pressure from both the students and the parents not to do that. We do not think we should be skewing choice in that way.
Neil Carmichael: I am going to probe that issue in a moment, but Charlotte wants to chip in.
Q471 Charlotte Leslie: Thank you very much, Neil. You have half answered it, Kate; you said something quite interesting earlier on. You said, "I want what is best for my child, not for them to be constricted by a textbook." Would you hypothetically-you have half answered it-prefer that your child had a greater indepth understanding of the subject, yet perhaps more risk of not doing quite so well in the exam because they had not been taught what hoops they were going to have to jump through, or would you prefer that your child had a more solid chance of jumping through the hoops of the exam and coming out with a better grade on their CV? Do you think that the accountability structures are at the heart of this, and even if we make structural changes around endorsement, because of the basic pressures in the system it would evolve back into something quite similar to what we have.
Kate Harris: That is a really good question, and I think my answer of course is "both/and": I want them to have the best, deepest, richest education that will prepare them for whatever they are going to do afterwards, and I want them to get the good grades too. I think teachers should be able to do that, and the job of publishers is to create resources that help teachers to do both/and, with a situation where choice is not skewed, so there is a plurality of choices and teachers can choose what are going to be the best materials for their particular teaching style and the learning style of the children they are teaching.
Q472 Neil Carmichael: Following on from that question, we know the accountability system exists in schools and parents are using it. Are publishers using it? Are publishers looking at the strength that accountability might bring to focusing the sales of books?
Jacob Pienaar: Publishers will be aware of it, simply because the job of publishers would also be to go out and engage with teachers and hear their needs, and also be aware of parents and learners, because ultimately we have to remember that it is the learners that are the endusers of our products, although they might not be the interface that we deal with. Publishers will be aware of it, and they would probably, in one way or another, try to reflect the particular need that sits alongside the range of other needs that I think your question has summarised to a certain extent. Our job is to try to combine those two things. In other words, what a really good textbook will do is do both. You can do proper preparation for examinations and you can bring the depth, breadth and richness of the wider curriculum to that textbook. It is not an either/or.
Pat Glass: Ian wants to come in on that.
Q473 Ian Mearns: At a previous panel, I suggested to members of that panel that the relationship between examiners, publisher, authors and individual examiners could be seen as being a tad incestuous-that was the word I used-and a member of the panel said, "No, it is much worse than that." Amusing, but serious, I think. I have a deepseated concern about endorsed textbooks narrowing the educational subject knowledge for some pupils. I have a concern, and I am going to ask you outright: are endorsed textbooks in the best interest of exam boards and publishers, rather than the interest of young people and their wellrounded education?
Kate Harris: It is important to distinguish the nature of different sorts of endorsements. Again, at one end of the spectrum, you have the same organisation publishing resources and owning the awarding body. At the other end of the spectrum, you have generic textbooks of the sort John was describing that are not linked to any particular specification. In between, you have endorsements that say, "This is useful for our specification," but not that narrow, limiting effect you were talking about, Ian. Where you have exclusive endorsements and/or that very much official textbook for structural reasons, there is a real risk of that narrowing happening. Where there is more plurality, it is less likely to happen.
At OUP we publish materials in partnership with OCR; for every partnership publishing opportunity for each individual specification they go through a very rigorous tendering process, where publishers come and say, "This is what we would do to support education in that area." A few years ago we lost a tender to one of our competitors, which I was very upset about, obviously, and I said, "Why did we lose it?" The answer from OCR was that the other people were more convincing about the enrichment they would bring to the publishing they were doing for that subject.
That was really interesting: they were not saying they lost the tender because they were not going to follow the spec, because following the spec is the absolute baseline; it is what you do beyond that. That particular endorsement went to a company that demonstrated to OCR that they were going to bring richness to those specifications-to the teaching for those specifications. We must be a little bit careful; all endorsements are not the same.
Q474 Ian Mearns: But I was taken by your answer to the earlier question, Kate-that what you want is that any child has a wellrounded and full knowledge of a subject, as well as getting the grades. Of course, when it comes to progression, it is the wellrounded subject knowledge that is going to be of more use to them, but they have to get the grades to get into institutions that they want to get into. Sometimes there has to be a dilemma there in terms of what we are producing in terms of the youngsters and their progression through the education system. We have had scientists at the panel telling us that they were concerned with the lack of rigour and inquisitiveness of youngsters coming into their higher educational institutions: their lack of inquisitiveness in terms of how they wanted to study their subject matter. Are we in danger of narrowing down youngsters and just turning out a factory for passing exams?
John Butterworth: Could I come in on that question? The subject I both examine and at the moment write in is critical thinking, which is supposed to do that very thing of broadening and inculcating in the student an ability to think about what they know, as well as just containing the knowledge, but the system tends to narrow even a subject like that down into short-answer questions and very prescriptive mark schemes, and then books that encourage students to learn just what they need to know.
I have a textbook here that says on the back, "Developed with and exclusively endorsed by AQA. This is the only series to provide teachers with complete reassurance that they have everything they need to deliver AQA’s new Alevel Critical Thinking." Although I wrote that book, I am ashamed to see that on the cover, because I do not think that. It borders on trade descriptions, because this book is neither necessary nor sufficient. I would hate to think that any student or teacher really believed it would give them all they needed to deliver the subject.
Q475 Neil Carmichael: Doesn’t that remark on the back of the book reflect the problem, and that is the essential focus of this line of questioning: are the books there to focus on the exam or the enrichment?
Rod Bristow: They should be able to do both.
Q476 Tessa Munt: There are a couple of things that I would like to ask. If you publishers all just carried on publishing and we had a national exam board that said, "We are going to take questions from right the way across this set of publishers," what would happen then? Say you have Mr Butterworth and various other people here all writing their books with their expertise and their knowledge, and that is on offer to the students, and then you have a single national exam board who say, "Right, we are going to choose questions from all of these recognised publishers’ books, but based in the knowledge that you would find by reading any and every one of their books."
Kate Harris: The nightmare scenario for me is a single national exam board with an endorsed or partner or same structural entity publisher.
Q477 Tessa Munt: I am not saying that there should be; I am saying that there should not be, so you do not get the situation where you can have the exam board or publisher teaming up. What you do is to say, "These are accepted experts in their fields of whatever it is; they are writing with expertise." Then you have a body that says, "We are not tied to anyone; we are just a national exam board. What we want to do is check expertise, knowledge and skill and all the rest of it, so we are going to select from across the board of all of you authors questions that appear in any and all of your books."
Paul Howarth: I think you will see specialisation. If you look at the three publishers represented here, I would say we have specialisms within certain subjects. Depending on what happens with the outcome of the exam, or the single national awarding body, you might see those specialisations becoming more acute.
Tessa Munt: How can that be?
Paul Howarth: For example, Pearson is a maths specialist; OUP has strengths particularly in history; Nelson Thornes has particular strength in the science curricula. I could potentially see that being exacerbated with a single awarding body, so choice could potentially be limited.
Q478 Tessa Munt: Okay, fine. Can I just ask you one other thing very quickly? It strikes me there is a bit of a conflict with reality, in that young people use the internet and we are still sitting here talking about books. It is a slightly old fashioned and rather nice concept, I know. I just wondered how you see things developing into the future. My children are certainly encouraged constantly to search and check on the internet for this, that and the other. I have absolutely no idea of the quality of a lot of what they are accessing. Do you have any comments on why we are still discussing textbooks?
Kate Harris: To be honest, it is a both/and. We are all using-all my colleagues here-textbooks as a shorthand for learning resources. No significant piece of publishing that I, Paul or Jacob did nowadays would not have a digital dimension. That digital dimension might be online, or it might be on disk, depending on where we think the schools are in terms of their technology, but we are using textbooks as shorthand for learning resources.
The point you make about knowing what information children are accessing is very important, and I would argue that the digital resources that Jacob, Paul or I are publishing have the sort of guarantee of quality that just jumping off into the internet will not necessarily have.
What I am advocating is that we continue to have choice, and that choice is not skewed by the relationship between publishers and awarding bodies-to have actual and perceived independence between the awarding bodies and the publishers. But that is not just for textbooks; it is around the digital resources too.
Jacob Pienaar: The question highlights an important fact: the richness and diversity of what goes into classrooms and into the lives of students and teachers at the moment should not be underestimated, and therefore we should maybe not overestimate. Of course, we like teachers and schools and learners to use materials that we produce because we think it brings a certain stamp of quality. But you are right: there might be very different perspectives on that.
We should recognise that, but it also highlights the fact that it is a very free market, and there is a lot of free choice out there. Whatever we say here, at the end of the day teachers can use whatever they want: whatever we do or do not publish, they can, for that very reason. There is not a system of quality control, other, interestingly enough, than the endorsements. I am not talking about exclusive arrangements: I absolutely agree we should be very careful of those. But regarding endorsements, there might be a bit of a misunderstanding. Endorsements are also used as a baseline quality check to ensure that, for example, learning resources do in fact match a particular specification. It might not be perfect, but it could make some sort of contribution towards quality control. But in the wider market, any of you can produce learning and teaching materials and sell them to a school, and that is the reality of the matter.
Q479 Ian Mearns: There has to be a buyer, though: to have a seller and a buyer relationship, there has to be a buyer. It has to be said that it is easy to say you can sell a book to a school, but there has to be an initiative for them to buy it in the first place.
Jacob Pienaar: Absolutely, I agree.
Tessa Munt: I would just suggest that, regardless of not having a written endorsement, such as Mr Butterworth referred to, on the back of a book, the visual message in the clarity of the colouring, the design and everything is quite enough to persuade somebody, I am sure, that it is a resource they might be using. Thank you.
Q480 Ian Mearns: I am taken by Kate’s nightmare scenario of a single examination board, because we have just been to Singapore, and that is one of the examples that we are being exhorted to look at and follow. They do have a single examination board run entirely by the Government, and it is an interesting thing that we saw there, but I think you can draw your own conclusions.
Kate Harris: The nightmare for me is not necessarily the single examination board; it is the single examination board with a single tiedin, exclusively endorsed publisher. Regarding this single examination board, in a way we all publish for Key Stages 1, 2 and 3, and effectively what we have at Key Stages 1, 2 and 3 is a National Curriculum and no exclusive relationships between some part of that National Curriculum and the publishers, and we publish in free competition with each other without choice being skewed in Key Stages 1, 2 and 3. It is workable.
I suppose for me, Ian, if you had that single national exam board, the risk is potentially the next step into an exclusive arrangement with creators of resources. If you were to do that, you really would have to guarantee independence between the publishers of resources and the single national exam board, otherwise choices for students and teachers would be ultimately limited.
Q481 Ian Mearns: One the lessons I learnt from visiting Singapore is we all have to be careful what we wish for.
Anyway, Jacob in particular: there is a relationship between Pearson and Edexcel in terms of its examination function. Do you think there is a conflict of interest between your examining and publishing functions? Are they too closely integrated?
Jacob Pienaar: As I said earlier, it is important to make a distinction between what the awarding body does in the area of specifications, which is linked to the curriculum, and what it does in terms of examinations. I would agree, and therefore in our particular case we have very strict firewalls between the people involved in textbooks and in the actual examinations, because we do agree that the risk there is high. It might not help towards that bigger coherence we were talking about and the quote from Tim Oates that was referred to, but we agree that it is important. We make that separation.
Q482 Ian Mearns: Do you understand the concern, for instance of a member of the Society of Authors who suggested that "publishers have stifled initiative in their pursuit of materials ‘more or less guaranteed to get pupils through the tests’"?
Jacob Pienaar: It goes back to what we were saying about narrowness, yes, and we absolutely understand the fears about narrowness and teaching to the test.
Q483 Ian Mearns: Would members of the panel accept that selfregulation has led to these concerns? Has selfregulation been successful, or do we need to reexamine it?
Kate Harris: May I come back on some of the things that Jacob was saying about the separation? While I completely accept that there are very strict rules separating the people who are marking and setting the particular exams from the publishing arm of Pearson, nevertheless, in terms of the spec development, there are very close links. For example, a job of head of science, an advertisement from the internet says, is "to lead and manage the science team on business strategy planning and the execution of green strategies"-blah, blah blah-"qualifications, resources and new business development". The specification development bit is very integrated with the publishing. Equally, this is an ad for a book, and it says in the advertisement, "This follows all the units of the specification." The specification has not even been published, so how can it? Again, it is about those official textbooks; it is about that inside track-if you are advertising a book that follows all the units of the specification when that specification has not been published, that is limiting choice, because it is the only book that can possibly know those units.
Q484 Ian Mearns: Kate, I could play devil’s advocate and suggest that you are just trying to protect your commercial interest by objecting to what Pearson do.
Kate Harris: Clearly, as you have heard, Ian, what I am looking for is plurality and choice, and the fact that teachers’ choice should not be skewed by exclusive relationships. If in free choice my books were the books chosen by teachers, I would of course be very pleased, Ian: I would be very pleased about that.
Q485 Pat Glass: Can I wind up this first section? Kate has very clearly said a recommendation for one single exam board is not one she would give. What about the rest of you? You said it was a worst-case scenario.
Tessa Munt: To clarify, I was not asking that; I was saying many publishers, one exam board. You said quite clearly your worst nightmare was one exam board, one preferred publisher.
Pat Glass: Specifically, leaving the publishers to one side, I am asking about the one exam board: what is your recommendation around that?
Paul Howarth: When I visit schools, talk to heads of department, deputy heads and head teachers, they like the fact that there is choice with the awarding body market. That is what they make their initial decision on, not the learning materials. That decision is made for 101 different reasons, depending on the school, its location, the subject and their position in the league table, and what ultimately they want to deliver to their students. I think they like the choice, so we should back what teachers and schools want.
Jacob Pienaar: We think it is probably possible to make the current system work, with some improvements, in terms of public confidence, but Pearson has also publicly stated that we believe that nothing is off the table and that we should consider the best possible system with the best possible outcome for learners in the UK.
John Butterworth: I have thought long and hard about this one; I am not really qualified to give a straight answer to it. I have worries that competition results in a gradual and almost inevitable-with the best will in the world-gradual inching down, or holding down, of challenge. But on the other hand, I can see the nightmare scenario aspect of having one exam board, particularly if it tended towards the Singaporean model, where the Ministry decided what should be published and not published in education.
Pat Glass: There are pros and cons in both systems.
John Butterworth: In a way the threat of having a single exam board imposed because of the danger of competition between exam boards is something that should make the exam boards very careful-and Ofqual too-about making sure that there is parity between the same award, the same subject, in different boards.
Q486 Pat Glass: So the current system is not perfect, it is far from perfect, but we need to be very careful about what we wish for.
John Butterworth: I would hate to see it go for the same reason I would hate to see only one publisher.
Tessa Munt: Damian has one last question.
Q487 Damian Hinds: Paul, you mentioned 101 reasons why schools made choice decisions between different exam boards. Can you give us the top five?
Paul Howarth: Gosh. I would probably struggle doing that. We work with a grammar school as a partner in Cheltenham, and they are looking at extension; they are looking at giving their top pupils a truly broad interpretation of the curricula and added learning on top of that. I have been in schools where we try to sell our textbooks and they say, "We cannot buy any resources; we are having to let half our department go." They will go for cheap resources, they will probably go for the best way of getting as many students to a grade A to C or equivalent that they possibly can to maintain funding or whatever it is it might do. For me, those are the two extremes; that is where choice comes down, and ultimately what I mentioned earlier on is the fact that a lot of schools teach to get the exam. That is a challenge, and we need to be aware of that. I know from a resources perspective as a publisher that we try to cover the whole gamut from a base text that helps people achieve those aims through to extension, and also for those people that are unable to achieve the base aims because they have learning difficulties.
Pat Glass: Thank you very much.
Examination of Witnesses
Witnesses: Rod Bristow, President, Pearson UK, on behalf of Edexcel, Mark Dawe, Chief Executive, OCR, and Andrew Hall, Chief Executive Officer, AQA, gave evidence.
Q488 Chair: Good morning gentlemen. Thank you very much for coming today and appearing before the Committee. We have had an interesting time: we had the scandal before Christmas, and you came before us then, so we have had the pleasure of seeing you twice during the one inquiry. Ministers say they are open to radical reform and dissatisfied with the way you have been conducting yourselves. Is that what we need: do we need a radical structural change? Andrew?
Andrew Hall: No, I do not think we do, but we do need to move from where we are. It is impossible to look at what has happened in the past and say things are not perfect. I was quite fascinated to listen to the publishers in the previous session. To some extent I think actions speak louder than words, so if I take it from that perspective, we have changed. When I joined the organisation less than two years ago, I enquired about why we have this relationship. Looking at it, it was probably the right decision my predecessor made five, six years ago to have that relationship, because there was not a breadth of resources for students. We did not have the internet in the way it is now. It was important to ensure something like the 150 specifications we offered were covered.
Times have moved on. The publishing industry knows, well before this inquiry started, that I made the decision we were not going to do that: we were going to work with all publishers and choose those that were best for each subject. We can have change. We also need to have change in the way we look to structure some of our qualifications. I have been quite vocal that we need to change the way resitting is done in our examinations. We need to look at the structure of Alevels, the weighting in A2: there are a whole lot of changes we need to make. Radical reform is a very dangerous thing to look at, if you look at the change that is going on at the moment. I should probably stop there.
Q489 Chair: Are we spending too much money on qualifications? Is too much of schools’ budgets going on exams, Mark?
Mark Dawe: No, I do not think so; it is about 2.5% of the overall education budget for a quality assurance system that-
Q490 Chair: The massive increase we have seen in expenditure by schools on exams, all the resits and the rest of it, is all money well spent?
Mark Dawe: There is more money spent; the students are taking more qualifications and papers, so the budgets have gone up. You might want to look at resits and say resits are now being taken for GCSEs, and that will make a difference. Going back to your point, I do not think we have a problem. We are still the envy of the world; Singapore look to us for our system and what we do. Singapore celebrates when exam results go up; they do not say, "This is terrible." But there is clearly public concern about the exam system and what is happening in schools, and that loss of breadth of learning. We need to put a solution in place to sort that. One of the ideas we have already come up with at OCR is the involvement of HE as the custodian of Alevels to ensure that you are getting that breadth and what HE wants. That is a one of a number of ideas we have put forward to resolve these issues.
Q491 Chair: But Singapore, of course, does not have multiple awarding bodies.
Mark Dawe: No, and Singapore is the size of the Isle of Wight with a population half of London and you get arrested if you drop chewing gum on the ground, so there are quite a few differences. We have to be careful about what comparisons we make.
Rod Bristow: This issue of standards and the need for change is key; I think there is need for change. We need to be careful about what we wish for, though, in terms of what that change is, but standards are incredibly important. There is nothing more important than the confidence young people have in what they are learning and what that will lead to: the credibility and integrity of the qualifications, and the reputation for quality that they have in terms of progression. That is why we have launched this consultation paper called Leading on Standards, where we have put out some very specific ideas about things that can be done to put an increased emphasis on standards. It is that focus on standards that is more important than the focus on the systemic issues-the system change.
Q492 Chair: Singapore told us they had not had a shift in the percentage of people getting A grades, for instance: they were pretty consistent over time. They had not seen the alleged dumbing down that we have. Might that not be a systemic issue?
Rod Bristow: There are many complex reasons why grades improve over time. There is no question that teaching and learning, the amount of energy and effort that goes into teaching and learning has improved.
Q493 Chair: But not in Singapore?
Rod Bristow: Well, we need to ask the question about the way the grading gets done. It is possible to mark papers entirely on a normreferencing basis, where, regardless of what gets achieved, the grades will not improve.
Chair: That is not what they do.
Rod Bristow: But the issue is that we have operated in a system that has focused on comparability and compliance with what the regulator requires. It is absolutely time to focus much more now on what the absolute standard is that we should be aiming for, to work much more closely with higher education and employers, to look more at international comparisons; those are exactly the areas that we are focusing on, and we think that the wider debate should be focusing on: "What are the things we should be doing?" Then we can address the issue of what is the best system within which they should be done.
Q494 Chair: But if one’s analysis is that the system itself is driving the behaviour that one does not want, changing the system might be the obvious solution. Mark brought up the issue of Singapore; Singapore does not have normreferencing as its central way of allocating marks. It has a single awarding body, and looking at the energy and drive and attention to education they have in Singapore, I find it hard to believe that their standards over the last decade have not risen with just as much energy and improvement as ours have, and yet they have not seen a drop in grades. Why should we not radically restructure the awarding bodies?
Andrew Hall: I am in a slightly different place to Rod on the standards point. It is a word that is capable of meaning two things, certainly: there are grading standards and content standards. In an earlier session when you had the technical people from our awarding body industry, they gave some very good explanations-nothing that I would fundamentally disagree with-of how, over time, ticking a second a year, there has been a creep in grading standards. We have to accept that. The awarding bodies in conjunction with Ofqual did something about that two years ago. That was made clear: "No, we are going to reference back the statistical predictions to the first year of the specification." That well and truly nails that point, and we have seen evidence of it in the Alevel results last year: it was very, very clear.
If you then look at the other part, the content standards, there is something where we as an industry-almost as a society-have seen a change in the content of qualifications. Again, I quite publicly said when I came into this industry-and indeed said when Ofqual was being formed, and I was the Accounting Officer in QCA and got into some trouble for saying it-that we needed a greater focus in regulation on content standards, not just on process. There has been change in content over time, and if we can start to really fix it-it goes back to the publishing point as well-we have to really be clear about the richness. I would like to see a change of broader syllabuses-syllabi, I think is the plural-to allow us to sample from a greater thing. There has to be a focus on how we develop content standards for examinations.
Q495 Alex Cunningham: So there has been dumbing down, as the Chairman suggests?
Andrew Hall: What I am saying is there has been a change in content, specification to specification.
Q496 Alex Cunningham: Has there been dumbing down or not?
Andrew Hall: I do not think that dumbing down is necessarily the right term.
Q497 Alex Cunningham: What would you use?
Andrew Hall: I would say there have been changes in contents. If you take computing Alevel, which is one I happen to sit on and am awarding, what is examined in computing now is very different from what it was five, 10 years ago. If you take mathematics, certainly when I did my pure mathematics I learnt a lot of proofs, and I could recite them; I would fail miserably now. Mathematics now has more investigation. It is about what the appropriate content is for where we go, and it is changing. We need to think about how we determine the content.
Q498 Alex Cunningham: But if there has been grade creep, as you have described, surely it indicates that people are getting a higher grade for a lower content, lower quality exam paper.
Andrew Hall: The point I was trying to make quite clearly is to accept that, in a short run, you would not see the difference, but over an extended run-for all the reasons that the researchers explained to you last time, and I could happily repeat if you want-when we used statistical predictions, examiners in the past have always given the benefit of doubt. Whether there is a grade on 56 or 57 UMS, it has tended to go one way. When I first came here I kept bar charts for the first year of which ones were up and which were down, and there was a change. Our response to that has to be going back to reference it to a base year so you tackle that issue.
Q499 Alex Cunningham: I get a bit fed up with the achievement of our young people being done down. Are you saying that the grades our young people are getting today should not be as high as they are?
Andrew Hall: No, what I am saying is that the wrong question is to compare the grades now with 20 years ago, because there has been a change.
Q500 Damian Hinds: That is what people do, and they expect to be able to do it: employers expect to be able to do it, and universities expect to be able to do it.
Andrew Hall: No, I would argue that they should look at the grades over a five to six-year run. When I look at my Alevel grades, I do not compare them with my children’s.
Q501 Alex Cunningham: So is an Alevel awarded six years ago worth the same as one awarded today in the same subject?
Rod Bristow: I would agree with many of the distinguished academics that you have seen recently at this Committee, who said very clearly that it is quite impossible to say what has happened-whether there has been this dumbing down. If you look at it, there is no objective way of measuring that. The thing to focus on-rather than having a long debate about whether it has or whether it has not at a time of curriculum change-is to look at what universities are looking for, to look into what employers are looking for, to look outside, at the international comparisons-to look forward and outside-and to put our attentions into making sure that the standards we have now are fit for purpose and are serving our young people well. I would say that is where we should be focusing our energy now.
Mark Dawe: Changing the exam boards is not solving the problem. Standards are the vital thing; we have to define what standards we want. We have talked about international standards, then standards between boards, subjects and years: you get to a point where you cannot have everything. We have to define that very clearly. It is wrong to condemn students and teachers because they are working hard to understand what the exam system is asking of them and achieving it. I am a big cycling fan. Twenty years ago we got no medals; this year we are getting loads of medals at the world championships. Is that because it has all suddenly got easier, or is it because they have been training harder?
Q502 Chair: We have the effort on that, the PISA tables, which suggest that while we are handing out the medals at home, we are falling down the league tables abroad. Singapore, which has phenomenal energy in its education system and is lifting performance across the board, has not seen a massive change in the number of A grades, for instance. It is slightly frustrating: Rod, you just said the academics are not too sure. With a big increase in the percentage of people doing Alevels, and a massive increase in the number of those of a wider pool, thus on average a lower quality pool than we had before when it was more of an elite pursuit, and a bigger percentage of them getting Alevels, it is quite obvious that an A grade at Alevel has got a lot easier. Trying to deny the obvious truth because it is attacking our kids-it is not-does not get us anywhere if we effectively fail to notice what is the obvious elephant in the room.
Rod Bristow: I am not denying it; I am saying we must recognise the issue and the problem. It is true that whilst actual grades awarded have been going up, our performance on international league tables has not. It is true that we hear from universities that young people do not have all the skills and knowledge that they need for the courses they are now doing, and we hear similar things from employers. I am absolutely saying there is a need for us to look at this; there is a need for us to move forward. We do have to take this incredibly seriously, but I am suggesting we really focus on those issues, listen to higher education and employers, look at what is happening in other countries and take specific actions. That is what we are absolutely committed to doing.
Mark Dawe: We are worried about Singapore because their young people will take the jobs that our young people deserve. We have to work out what our young people should be learning; it is not even exams, but what the breadth of their learning is, the exams that fit that, and then the progression to university and employment. It goes back to the point that if universities are happy with what happens in the sixth form and beforehand, we should be happy with what is going on in the schools and the exam system that is supporting it.
Q503 Ian Mearns: So you believe it should be as in Singapore, with a regulated and centralised system, where their Ministry of Education has an ethos that says, "We want to produce young people who are going to be of benefit to the economy." That is one of their top straplines.
Mark Dawe: Universities have a wide range of benefits they think they are contributing towards: economic, social, there is a range of things. We should listen to the experts in the universities. OCR-I think we have given you a factsheet on this-have 10 subject communities with over 100 university representatives. We are listening to them and understanding what it is they like, because they like a lot in the existing system. We must not forget that we do have a gold standard qualification-maybe we are looking for platinum now or something-and they do like a lot of it, but some bits they say are missing. It is often not the knowledge; it is the skills. That is common across all the subjects. It is not a knowledge issue; it is the critical thinking, the solving, which can be applied to every subject. That is what we are getting feedback on, and that is the sort of thing we are starting to work on: how can we embed it in our qualifications and ensure that the learning embeds it as well?
Q504 Damian Hinds: You are competitors, all of you. Mark, what is your elevator pitch on why you are better than the other two?
Mark Dawe: My elevator pitch for OCR is that we are a not-for-profit organisation. We are part of a university; we are part of a larger group that operates in 150 countries, which allows us to have a research team of over 60 researchers, so I can put ideas to them and understand what works and what does not work. We are at the core of the education system; we work within the education system, and have done for the last 150 years.
Q505 Damian Hinds: Thank you. Rod, why is your maths GCSE so popular?
Rod Bristow: Of the different GCSEs, maths is a very interesting subject, because the specifications between the awarding bodies are very similar, given the nature of the subject and the way the subject criteria are laid down. The reason why maths is so popular for Edexcel is because of the amount of support and the reputation for giving really good support to teachers in understanding the specification.
Q506 Damian Hinds: Thank you. Andrew, why is your single award English so popular?
Andrew Hall: It comes from a lot of history, when AQA many years ago created an anthology to provide lots of the set texts in one go to schools to save them buying lots of books. That became very popular, and that took on strength. That was many, many years ago, and from that, there have been changes. If you look at our share-I am not necessarily proud to say it, but I will here-it has decreased over time as people, particularly from Wales, have come and taken it.
Q507 Damian Hinds: I was just coming to that: how do you feel about that?
Andrew Hall: Mixed, I think is the truth.
Q508 Damian Hinds: Why has it happened? Why has WJEC stolen your lunch?
Andrew Hall: I do not think they have stolen our lunch. For me it is important that the students that take an English exam with AQA, or with these other guys, are capable of going on and progressing. When I enquired, having come in, it was quite fascinating; I do not claim the new boy thing at two years, but you do ask some questions when you come. I asked why this happened, and the anecdotal explanation I was given was that there was a time when the Welsh subject had significantly less poetry content in it, and a number of people moved at that time. Whether that is the reason or not, I do not know. That is certainly not the case now, but there was a move then.
Q509 Damian Hinds: I was at a JNB school: all my Olevels came on the same certificate. I have no way of knowing if that was common, but my assumption has always been that when I was at school, generally speaking you went to a school that had one exam board. That may be right or it may be wrong, and apparently the DfE records do not go back far enough for us to be able to tell. But we do know that today only one in seven schools use a single exam board for at least four-fifths of their subjects. Why?
Mark Dawe: It is down to teacher choice. You asked already why teachers change. When we talk to them, we are not talking to the principal or the head; it is the heads of department. If the teacher changes, they have a preference for what they want to deliver to the students and how they want to deliver. I was in a college the other day and asking, "Why don’t you do our maths Alevel?" Apparently, integration and differentiation are in two different units in our qualification, and he wants to teach them together. I think it was AQA spec that put them together, and that was the choice. There are a lot of matters. Also, some of them have had bad experiences with exam boards, the admin and how we treated them, and they want to change. It is good that they have that opportunity to change and show their dissatisfaction.
Q510 Damian Hinds: Andrew and Rod, in the historical experience of your organisations, has there been a move towards schools taking on more, different exam bodies? The mean number of exam bodies per school is now over three, and as there are only four exam bodies, that is quite a lot. Has there been a change over time?
Rod Bristow: The interesting thing is that if you look at market share over time, it has been reasonably consistent.
Q511 Damian Hinds: But there has been movement between the subjects.
Rod Bristow: There has been movement between the subjects, which tends to happen at a time of curriculum change, when new specifications are issued. Very much for the reasons that Mark was outlining, teachers will then look at those specifications and work out which ones are the most suitable for their style of teaching or how they perceive the needs of their students. One of the things that has changed is that if you were to go back 20 or 30 years, most people would say definitely that different exams from different exam boards were either easier or more difficult, but the standards were not the same. That is not the case today. There is a general acceptance that the standards between the exam boards are very close.
Q512 Damian Hinds: Would it possibly be the case-speculating entirely-that exam boards used to have a suite of exams in which some subjects may have been easier, but that today it is more possible to pick and choose, and for each subject there is an easier exam board?
Andrew Hall: I looked before I came today at some of the churn, which is the word I would use: change in schools and colleges taking subjects with us. If your contention was right, you would see a general direction to it. If I take geography GCSE, for example, for which we awarded the spec for the first time in 2011, 27.6% of the schools taking that with us had not taken geography with us in the previous year.
Q513 Damian Hinds: What was it?
Andrew Hall: 27.6%. So we would say we won; 27.6% of the schools were new. But, to make the point, of the schools we had the year before, 21.3% went to other boards. There was a real significant change.
Q514 Damian Hinds: Where do they end, at the end of this journey?
Andrew Hall: We stayed about 4% up in total, but year on year something like 2% of the schools that take any subject with us, in the course of normal, no specification change, move away, and something like 2% or 3% move in, and there is that natural churn. When I look at the reasons why I believe that is, top of the list is generally dissatisfaction with the quality of marking, or they did not like their results. It is normally a dissatisfaction with us that has caused them to go somewhere else-that is the cause of change. The big change happens on curriculum review. We are just about coming up to a new national curriculum review. There will be change. You asked earlier about time, and my message is we need to be really careful about the rate of change we introduce in the National Curriculum. Beware of officials saying, "We need to get this done very quickly," as you get poor product.
Q515 Damian Hinds: Mark, your 21st Century Science suite is said to be "proven to help improve grades". How does that process happen?
Mark Dawe: Again, we worked with the science community to create that qualification. It was incredibly popular, and the resources that went with it were incredibly popular. It raised the number doing science, not just shifting to OCR; it raised the number that went to university doing science.
Q516 Damian Hinds: How does that improve grades? I understood part of that, but not all of it.
Mark Dawe: The universities then took those scientists and were happy with the achievement of those scientists, and they went through, did their degree, and UKCES is saying the number of students coming out with STEM qualifications now is pretty much meeting demand.
Q517 Damian Hinds: Sorry, that has increased the numbers, rather than improving the grades.
Mark Dawe: Both happened.
Q518 Damian Hinds: What is the process that makes the grades improve? What are the key factors that mean the grades rise? I understood part of what you said: it is more interesting, it is more exciting and the content is more relevant. Is that it, or is there more?
Mark Dawe: The teachers understood how to teach, it gave them a better opportunity, a different approach to science, and that clearly worked for the teachers and the students.
Q519 Damian Hinds: Thank you. Rod, Pearson told us before that commercial pressures create pressure to raise standards rather than to lower them. How do those pressures manifest themselves?
Rod Bristow: There are some specific examples I can give you. For example, last year, we announced that we were launching a new generation of our vocational qualifications in schools, the BTEC. We have essentially raised the bar: we have introduced external assessment; we have put a huge amount of extra quality control in there; we have integrated numeracy and literacy skills in there, because we had heard from higher education that these were concerns for people progressing into higher education. The reason we did it is because it is incredibly important to us that the BTEC has the reputation that it deserves, that it is seen by higher education, seen by employers, as something that is highly credible.
As a specific example-there are other examples-we have a relationship with the University of York; we have an Alevel biology qualification with Salters-Nuffield that we have worked very closely with the university on. It is highly credible, and our competitors have similar schemes, with Cambridge PreU and the AQA Baccalaureate. There are many examples of innovation around raising standards, recognising that is what young people want and that is what their parents want-they want them to have highly credible qualifications. If we can fulfil that need, then we will be more successful.
Q520 Damian Hinds: Can I just come back to finish on the terms of competition again? Rod, we heard from you on 15 December that the key terms of competition between you and your colleagues here are "price, service and support". We know this is an oligopolistic, rapidly growing market, so to any student with Alevel economics it will not come as a great surprise that you do not seem to compete very much on price. Training courses, as we heard in the previous sessions, are not commercially priced but run as a not-for-profit exercise, or even crosssubsidised by your other activities. That leaves service and support. In what sense is service and support not another way of saying, "We’re helping you improve your grades"? If it is a more general version of service and support-like dealing with phone queries promptly and delivering the scripts on time and stuff-why have you managed that for some subjects but not others?
Rod Bristow: First of all, if I can differ on the point you made that price is not a factor, we do monitor the prices of our competitors’ qualifications very closely. We have one specific example from a few years ago where we got it wrong with our Alevels, and we subsequently lost a considerable amount of market share, because the prices were substantially different from those of our competitors. I do believe price makes a difference.
In terms of service and support-in some ways this relates to the conversation you had earlier with the publishers-one of the challenges teachers have is in support for delivering the specification for delivering the curriculum. It is right. Tim Oates has cited the research from Schmidt and Prawat about the importance of curriculum coherence. This is internationally accepted research that shows that if you do have that integration between the curriculum or the specification itself and the way in which it is delivered, you are going to get improved education, improved learning outcomes. We do that, and that is one of the reasons why awarding bodies have relationships with publishers; it is one of the reasons why we ourselves have relationships with many publishers. One of the things that perhaps did not come out of the earlier session is that we do not have exclusive relationships with any publisher; we encourage as many publishers as possible to support our specifications, and we do it for good reason. The more publishers that support our specifications, the more teachers are going to feel supported-that they have that support when they are out there delivering the specification. These things do line up, and these interests do have a positive outcome on learning.
Q521 Damian Hinds: We learnt in an earlier session that none of you makes a lot of money out of textbook endorsements, as far as we can make out, but what you have just said would not at all be inconsistent with seeing the textbook market, as it were, as a way-I am not saying this is what you were saying, but it is not inconsistent-of supporting making exams that much more accessible to the candidates taking them.
Rod Bristow: Let me distinguish between the exam and the specification. It is not the exam that is made more accessible: there are incredibly strong firewalls between exams and any other activities. The course, the specification, the curriculum and giving support for teachers to deliver that is the thing that-
Q522 Damian Hinds: But that is internal in your organisation. If I am a head of department I am seeing the two as two sides of the same-
Rod Bristow: But there is a very clear distinction.
Q523 Damian Hinds: Andrew, Mark, do you have anything to add on what service and support mean, other than helping you to improve your grades?
Andrew Hall: I have taken an increasingly clear view that it is more of a negative than a positive impact. I have talked about several things here: quality of marking-the confidence that teachers have in the marking, and bear in mind it is teachers that mark all of the exams. There is also the way that we deal with enquiries about results-those involved in education will be aware it is an incredibly sensitive time-and the way we get results out, the fact that we now send them electronically, and the way we deal with queries.
Q524 Damian Hinds: But forgive me, you would do that for all subjects presumably, and yet you have a markedly different record of success on some subjects: why is that?
Mark Dawe: It is often the story that you make a mistake in the specification. In one subject-it was design and technology-we talked to a lot of teachers, and they said, "We would really like it if it was like this," and we changed it and made it like that. We managed to still fit it to the criteria, and then they all walked away, because they did not like it. You do make mistakes along those lines, and that is something you have to correct. The big issue is they want the right results, and most teachers are sitting there, ranking their students and saying, "Roughly, this is the order we think our students should be in." When they do not get that ranking, they get very concerned, because they know those students. It comes down to the breadth of learning in the classroom and then giving the students the right results.
Most complaints I get when I am going around are, "Our biology students were all skewed." But that is one class in an enormous college; the rest they are quite happy with, and sometimes they say, "We will give you another chance," or they put an enquiry in and they get what they believe. Other times they walk away and say, "We are not comfortable with what you are doing; we are going to go to someone else," which is, again, the competition between us that allows them to move if they are dissatisfied with what they think is the quality of our marking.
Andrew Hall: If I can just come back on one point: ideally I would love the quality of service to be the same between each of our subjects. In any organisation that has the number of people we do, we have different strengths and weaknesses. Hand on heart, for transparency, we are better in some parts of our organisation than others. It is something we need to fix, but that is what we are doing. I think the others would be the same.
Q525 Chair: All the prices you charge seem to be about the same. If you were looking at a description of a cartel, it would be a small number of dominant players who, if they ever mistakenly put a price out of line with the others, learn not to, and then they all move as one. You look remarkably like that, from a price point of view. Are there some aggressive price competitors in the qualifications market that I am not aware of who offer a much lower price? A can of beans is a fairly standard product, but the prices for which a can of beans is available vary massively in the marketplace. Qualifications do not seem to vary massively in price.
Mark Dawe: Three years ago when I was a college principal I was interviewed by QCA, I think it was, and I said it was the most outrageous cartel I had ever seen. Now I am sitting on the other side.
Ian Mearns: And it still is.
Mark Dawe: We make a very small margin overall as an organisation, and we do look at prices. In GCSEs we are the cheapest; in Alevels, in the majority of cases we are the cheapest. That is one of the things about being a not for profit, and trying to bring the prices down as low as possible. In the vocation market it is far more competitive, and there are different margins that you claim. But in GCSEs we are also supporting a lot of minority subjects, so even though there may be money made because of volumes in one, there are significant minority areas like classics. I was looking at Classic Civilisation the other day: the entries as a whole would not sustain that GCSE, and there are 10 option papers in there as well, which makes it even more expensive to run, but we feel as an education charity it is our duty to provide that wide range of subjects. There are 2,000 students that benefit from that.
Q526 Chair: Can I pick up on a vocational qualifications issue? The Government has just done a massive cull of the vocational qualifications that count in the performance tables. Rod, just last year-funnily enough around the time that Alison Wolf came out with her report saying that a lot of qualifications were not worth the paper they were written on-you decided to raise standards in your qualifications. Was BTEC part of the problem that the Government and Alison Wolf identified?
Rod Bristow: Alison Wolf included a comment about BTEC in her report and how highly respected the BTEC qualification is. We have a lot of evidence about the economic return you get from that. We also have evidence in higher education of students who have done BTECs and how well they do in higher education. In fact, the thing that surprised me is that there are many, many courses in higher education where BTEC students will do better than students who have Alevel. I was surprised, because often the perception is that students who are less able will do vocational qualifications. That is less and less the case. We took action on BTEC well ahead of any Government decisions about what would count for school league tables. We just think it is a really important thing to do.
Q527 Chair: How many of your BTECs were affected by the recently announced Government cull?
Rod Bristow: Virtually all of our BTEC qualifications will have been accredited, effectively: they are now eligible for funding and will count towards league table points. We are, however, going through a process right now for the new generation of BTEC qualifications, which we will launch in 2012, but they will become official from 2013. We are working very hard to make sure that students continue to have access to BTEC in schools.
Q528 Chair: Sorry, just to be clear: you have just said that the vast majority will still count towards the-
Rod Bristow: Yes.
Q529 Chair: So BTECs were not really affected by the-
Rod Bristow: No. That is right.
Q530 Chair: What qualifications were affected, the 3,000, whatever it was, coming down to 120-odd?
Rod Bristow: This is a really interesting point. I doubt that the changes that were made will make a substantial difference to what happens in schools. The kinds of qualifications that were cited in the media at the time-things like fish husbandry or nail technology-were not necessarily the bulk of the vocational qualifications that were being delivered in schools. The qualifications that are there and accredited are the ones that schools were tending to choose in any case.
Q531 Chair: Most of your income comes from vocational qualifications, doesn’t it, Mark?
Mark Dawe: No, the opposite, and I have a slightly different view. Again, having come from a college, there are a lot of local schools running NVQs in this, that and the other and stacking up league table points. It was not appropriate: NVQs were not designed to be run in schools. In some of our collaborative work, we were doing NVQs at the college for the schools. I do not think that was not appropriate. I think a large amount of what has been taken out were the NVQs. The Cambridge Nationals and the BTECs are the predominant qualifications that are left on the list. There was slight tweaking required to meet the Wolf criteria, but only very slight. They are good substantial qualifications. We have the Cambridge Nationals 14 to 16 focus, and as Rod said, the dominant player in 16 to 19 is the BTEC full-time. I do not think that is right, and that is why we are introducing the Cambridge TEC, to bring in some competition. Again, it is a full-time, good, solid qualification that leads to progression. That is what it is about: it is giving students progression opportunities. Qualifications have no value in themselves; it is what they lead to. Is it a job? Is it progression to university or into further learning? If you can prove that is what they give you, they have some value.
Andrew Hall: If you had asked me 12 months ago I would have said it is not the last change. The much more important change that is being tackled is the equivalences between the qualifications, because they were clearly in my view-and in my organisation’s view-out of kilter. The fact that one of these gentlemen’s qualifications-be it a National or a BTEC-is no longer equivalent to four GCSEs but to one is a major step forward that we are really supportive of. I might stand accused of protecting commercial interests, but no-I will certainly speak out for what is right for students.
That equivalence is now in ranking; it is important, going forward, though, that we make sure there is some equivalence in terms of standards within there. The job is not 100% done.
Q532 Chair: What is your view on the equivalence changes, Mark?
Mark Dawe: It was the right thing to do, and the standards are there. It comes back to the one thing we have not really touched on: Ofqual’s core role should be around those standards and comparability. That should be their focus. If they do that properly it gives us all the approval, in a sense, that our qualifications are appropriate, and it gives the public the confidence they are looking for. At the moment we are in this vacuum where we have not got Ofqual being very clear that "yes, we are very comfortable with that", and doing the work that supports it and having the evidence that supports it and supports the whole system.
Q533 Chair: For the multiple awarding bodies system to work, we need a strong and capable Ofqual?
Mark Dawe: For any awarding body system to work-if you have one awarding body-you need someone to look at that standard. There has been a lot said about comparability between the boards; we can all be very similar, but at too low a level. The standards should be about where we want to pitch the level as well.
Q534 Chair: Just for the record, Rod, do you also agree with Andrew and Mark that the Government’s change on equivalences was appropriate?
Rod Bristow: I do agree. I support those changes, and I also agree with the point about the need for a strong regulator, and also a really collaborative debate about these issues and how we can ensure that we have incentives in the system that will drive up standards.
Q535 Alex Cunningham: You have all acknowledged the need for reform in the exam system, but I detect the view that it should be minimal rather than comprehensive. What evidence do you have that would suggest that fundamental reform is not necessary?
Mark Dawe: My view is that-we touched on this in the last session-it is around the transparency and the narrowing down. The concern seems to be the narrowing down. In some cases that is right: there is not the breadth of teaching going on because there is this targeting of exams. We need a debate about what it is we want in the classroom, and then how we achieve it. For the last 10, 20 years there has been an attempt to create much greater transparency, so the specifications are available to everyone-sample assessment materials, examples of question papers, examples of question answers-and the seminars, because we are trying to give every teacher the opportunity to understand what the exams are asking for. Twenty years ago people became examiners to find out. It was a lot less transparent, and you went into the exam hoping your question came up. Now there is much more of an understanding of what sorts of questions will come up. We need a debate about where we draw that line. That is the most important thing. It may involve some substantial change in how the central criteria are written-I will go back to my point about HE driving a lot of that-and then what that means coming through into the exam papers as well.
Q536 Alex Cunningham: So minimal or comprehensive change, Andrew?
Andrew Hall: You have to be really careful what you look for. My first leadership role in education when I moved out of industry was being asked to step up to sort QCA out. For those that were around when the SATs went wrong in 2008-which was, interestingly, a single awarding body model; I would draw attention to that-I found some things there when I was doing that that would make me steer very clearly away from a single awarding body model. The risk it concentrates in one place, the ability to respond to change, and the complacency it can create is very dangerous.
Q537 Alex Cunningham: So you do not think it will eliminate some of the issues around standards and comparability.
Andrew Hall: It would tackle in some ways the public perception that we three compete on standards. If there was only one, we could not be competing because we would not exist. It would help public perception, but it would bring very real system risk. It would put everything very close together, which may or may not be a good thing; that is a matter for debate. If you look at some of the other models that have been suggested-the franchising, one awarding body, one subject-that intuitively is more attractive than one awarding body, and you might again say "protecting interests," but it is not that. It has less risk to it, but it still has some risk.
Again, I go back just to give examples of "be careful what we get". When I was sorting out that National Curriculum test issue, I turned to my good friend here to help me redo it, as it happens, because Edexcel ran in. The reason that national awarding body got in a mess was because it wanted to change contractor for delivering at the end of the contract. There was no viable competition available because people had moved in different ways. The change was made, and then it cost an absolute fortune to put it right.
At the same time, you were not driving innovation and change. There is a menu here from which we can choose what we want. There is some real innovation that the competition drives: we have taken a view as an organisation that we really think there is a lot we can do with mathematics, so we have delivered some stuff to sit alongside Alevels to help students who want to study maths post-16. That would not come in a franchised environment.
The train example is almost where you get to: you get to the lowest cost service delivery for a tender and a massive infrastructure to decide how you create that-it is almost like QCA will come again, if we are not careful. There are real risks in either of those. The safest thing for delivering the most secure improvements for our students is to really tackle the content standards, which is where I started.
Q538 Alex Cunningham: You would agree that the idea to concentrate the best examiner expertise, particularly in shortage subjects, would work well within a franchise system?
Andrew Hall: I have a really different view on concentration of examiners. There is a real challenge that the country faces with the ageing nature of the examining population, because of the structure we created. What we need to be doing is finding a way to redefine the role of an examiner.
If you look at it now, it is like a pyramid: you have a chair of examiners, chief examiners, principal examiners, moderators. The burdens we put on those people to achieve what Mark talks about-bringing people out of HE and getting them involved-mean we need to redesign the roles of those examiners to make it possible for good teachers out of schools who have best current practice to give some of their time and come and be an examiner, and for university lecturers to give some of their time. If we concentrate the expertise into a few with a full-time job, there will be no progression. That modernisation is really important. I would say go the other way.
Q539 Alex Cunningham: Rod, would a stronger Ofqual be preferable to reform of the system? Assuming that is the case, in which ways?
Rod Bristow: To answer the question you asked my colleagues as well-is it minimal or comprehensive?-it is significant and we should be looking to embrace significant change. We should really take this head-on. It has to be the right change, of course; I will come to that. We have put together a paper that we have called Leading on Standards. We have contributed six ideas to the debate-some things we think should be looked at. I am sure there are others as well. For example, we have suggested setting up an independent standards board. We will do this ourselves; it could be done across the piece, but the remit of that board would be specifically to be looking at the future and looking internationally at whether we have the absolute standard right, rather than constantly looking just at things like comparability. The independent standards board is one idea.
Q540 Chair: Because you do not trust Ofqual.
Rod Bristow: It is not about not trusting Ofqual; Ofqual do a very good job on the issue of comparability. They will do a better and better job at that. There are probably other things that Ofqual could do and we could work with Ofqual on.
Q541 Alex Cunningham: What? What could they do?
Rod Bristow: For example, this issue around examiners: there are sensitivities with examiners. Examiners are excellent people: they are teachers, they bring real expertise, but I think with all the intensity of focus that there is on the system, the pressures on the system, and as we saw from the Daily Telegraph investigation, sometimes our examiners can be put into very difficult positions and some of the frailties there are exposed. Is there a way that we can collaborate together with Ofqual among the awarding bodies to make sure that we have some much more robust ways of working with examiners? Should we be looking to give examiners professional status? We have had some conversations with the Chartered Institute of Educational Assessors about that. I have written to my colleagues; I have had a conversation with Glenys Stacey about getting a discussion going about some of these issues that are really important.
Mark Dawe: We have had a standards board for 10 years, and we have had an Assessment Governors Board for 10 years doing what I have described. It is good to hear they are doing that as well now. As Cambridge Assessment, we raised the issue of standards two years ago in a debate, which was then followed by Ofqual as well. As I said, we have 60 researchers working on assessment and qualifications. That is their focus. I will pick one idea that Rod has put forward: the golden question. That was something I floated with our researchers a while back, and I have a big paper about why it will not work. I am happy to share that with you. You have to look at these ideas, but you also then have to challenge them and look at the evidence behind it.
Q542 Alex Cunningham: What needs to change, though, Mark? What needs to change? That is what I am interested in. We are all giving plenty of anecdotal evidence, but what needs to change?
Mark Dawe: One thing that needs to change is we need to understand what we want happening in the classrooms day to day. That is what we want to achieve. It has nothing to do with the examination systems to start with; it is the breadth of learning in the classroom that is the concern. Then we need to make sure that the exam system supports that, and a lot of that at the moment is driven by detailed criteria that have been set by central Government quangos. We have the opportunity to break that now; as I said, using HE or something else, we have the opportunity to break it. That is what needs to change: define what you want in the classroom and then how the exam system could support that.
Q543 Alex Cunningham: National subject committees as suggested by SCORE, the Wellcome Trust and Cambridge Assessment would contribute to that. Would that be a good idea?
Mark Dawe: National subject committees are a good idea. It is important, and we believe that is our business as an awarding body. We have always consulted with people; we create subject committees as well, because it is important to have a range of options, again going back to what the teachers might want and then choose from, with someone driving it.
Q544 Alex Cunningham: But could a national direction from a national committee be-
Mark Dawe: The problem is we have had that in the past and we ended up with what were the QCDA criteria, which caused a lot of the problems we are suffering today.
Andrew Hall: At the risk of competing with Mark for who has had a standards board for longer, AQA has also had one and it has a research committee to dream for, because it has most of the national experts on assessment, and people from places like America, Sweden, Netherlands-not Singapore, but we will come to that.
I did a jig the other day; I am not prone to excitement about these things, but Ofqual have created a standards advisory board for their organisation, and if people have had the misfortune to hear me talk in the press or publicly, they will know that a greater focus in Ofqual on standards is a really important thing. The fact that they are now taking that is really important. Interestingly enough our research committee is going to be decimated, but heyho, that is for the national good if people go there. It is a far better solution than awarding bodies setting up some sort of dialogue, because it will have greater public confidence and it will bring in broader stakeholders.
Ofqual is, in fairness, on a journey: the role of accounting officer in Government will be understood here. When Ofqual was formed I was Accounting Officer for QCA. I had to ask whether it had sufficient capacity. My answer at that time was that it did not have enough to tackle the standards issue. I have been encouraged to see the new regime of Ofqual focusing more and more on standards. It is on a journey, and that is a really important way that we will be truly regulated in that area, in a way that perhaps there has not been that technical understanding before. That is meant to be helpful.
Q545 Alex Cunningham: Has the current system really delivered innovation, though, that supports high quality teaching and learning? Why have some parts of the system-such as question paper setting-been relatively untouched when it comes to innovation?
Mark Dawe: The innovation is there, and it is happening. There are three areas of innovation that I can think of: one is the curriculum innovation. When we talked about science, we developed MEI Maths working with organisations to come up with something new and innovative. There is the IT side-the technology supporting the examining process-and enormous innovation there and enormous expense, particularly around electronic marking and the control it gives you over the standards of marking live time. We are investing tens of millions at the moment on systems around question paper production, and it is part of our discussion with Ofqual. These are the things we have done in the short time to try to prevent errors, and this is what is happening in the medium term. It takes time to put these things in place, and it takes an enormous amount of money.
Q546 Alex Cunningham: So you like it as it is, or there is still a long way to go?
Mark Dawe: There is still a lot of investment to go.
Q547 Alex Cunningham: You said investment; I said "a long way to go".
Mark Dawe: The "way" takes money, and so the money that we have available, we will continue to invest in those improvements. It is a top priority for us to get that into place.
Q548 Alex Cunningham: If online standardisation makes for "more rigorous, proven standardisation", as Andrew Hall suggests, then why are examiners not more enthusiastic about it?
Andrew Hall: I would suggest that some of them are less enthusiastic about it because it calls them to account. There is the good and the bad in this, and it is a more robust way of challenging and making sure progress is there. Something that gets lost in the process, and one of the things we have been considering, is the social side of a group of people who are interested in a subject being together to discuss the subject. That is something that has been lost through online standardisation. Recognising that and creating the communities to bring those people together-not for the purpose of standardisation, but to have their subject dialogue for their own professional development-is good. But it is that loss: that it is more challenging and demanding, but it is more accurate. The research evidence is absolutely clear that this makes for better quality of marking, which therefore makes for better assessments for our students, and the students getting the right results on the right day should be absolutely at the top of that.
Q549 Alex Cunningham: Your colleagues are nodding, so I will not ask them to address that question. If evidence suggests that the quality and reliability of marking has improved, why is that not reflected in increased confidence amongst teachers and the wider public?
Andrew Hall: I would argue it is because people now pay far more attention to the outcomes. It was always high stakes, but by golly, it has got more and more high stakes for schools and for students. There is more challenge, and we are more transparent. Students can now get access to their scripts so teachers review them in a way that never happened before. It is not perfect; the statistics say it is not perfect, which is why you have heard me talk about quality of marking, and those that are incorrect become more apparent and people remember the bad and not the good, and they should, and they should challenge us on that.
Mark Dawe: It is good to have enquiries about results, because it catches things we might have missed, but at the end of the day only 0.1% of grades are changed because of an enquiry.
Q550 Alex Cunningham: Finally from me, have developments such as online marking and standardisation led to cost savings for exam boards? I know you need the money to invest. Rod?
Rod Bristow: Yes, we have invested in those changes, which have improved our ability to invest. Certainly, in our case we invest very significantly in education. It is important that we invest in efficiencies that do not just improve the availability of funding to reinvest but also improve quality. With technology, we have shown that we can do that.
Q551 Chair: Could you give us briefly your reflections on the accountability system, which is an abiding concern of this Committee? The obvious one for secondary schools is the five good GCSEs, and we had the Secretary of State here a couple of weeks ago. You provide the qualifications: do we need a more balanced scorecard and what do you think it could look like?
Rod Bristow: I would say that we do. It is another thing we included in our Leading on Standards paper. We must not forget that qualifications are very high value, high currency. We should not be deceived into thinking that all these additional pressures coming in are all the fault of the school accountability system. Having said that, regarding a single measure of accountability-in effect, there is a single measure of accountability-if you look at the OECD studies of the most successful education systems, you will see that the most successful systems do tend to have a more rounded view of school accountability. In particular, thinking about progression: what are these qualifications? What is happening to young people when they have gone through their education? What do we know about that? In fact, if you look at the way that school accountability is approached in the independent sector, they take great pride in providing that more rounded view of education. That is something we should really embrace for all children in this country.
Andrew Hall: It is stating the obvious that any measure will have intended and unintended consequences. The idea of having no accountability would be abhorrent to me. What I have been really attracted to is not just the measure of five good GCSEs but the value added measure that alongside provides really meaningful data for people. How that gets prominence, time will tell, but we are going to have to have a measure, and it is about what you have alongside it to counterbalance it. In the end you have to reflect on the pressure it creates on the teaching community. Sometimes you can say, "We deliver the exams; some of the behaviours, are they ours, or are they people under pressure?" There is something about just how intense the pressure has been.
I went to Singapore about a year ago, ostensibly to look at how they use electronic assessment, but in the end I spent three days on a Singapore teacher training course, looking at how they develop formative assessment skills in teachers. The real investment in that teacher development in formative assessment as a tool for our teachers would help some of the pressures that the accountability measures create. It would ease the pressure-just a thought.
Mark Dawe: As a parent, I would worry if my child’s school was not getting five A*s to C for their students. So as a baseline, 50% five A*s to C does not seem unreasonable, and including English and maths. I worry about the other 50% who have not got five A*s to C. In a way we should be focusing on them, not the ones that are achieving it. A breadth of measurement always helps, because then you are looking at a number of things. But in English and maths, there is a fundamental problem that we need to sort out, particularly with maths. We have in our maths council and maths forums some ideas about broadening the time in schools, maybe, with two maths GCSEs rather than one, similar to English language and lit. That might be a way of doing it. It goes back to the education again, and what we are looking for.
But the other thing I worry about is the way we award at the moment and the rules we have to follow: we are never going to reach 80%, 90% getting English and maths A* to C like Singapore because our awarding system does not allow it, because we are fixing a curve each year predicted on the students’ ability. That restricts us. It does not matter how brilliant the teaching is for five years. At the moment, our system drives us to a set of A*s to Cs. We will not reach that; there are a number of issues there.
Rod Bristow: We recently carried out a survey of parents, asking them what they looked for in schools. It was quite interesting: the five A to C league table came out about number three. They are much more interested in, "Is this going to be the right school for my child? Will they be able to deal with issues that my children have and provide my children with the education that they need and that is relevant to them?" It is to do with things like quality of teaching, as you would expect. There are grounds for having another look at the accountability system to see what other measures could be included.
Q552 Chair: Do you have people with expertise to help in this regard? We are chewing away on this particular issue; we are quite clear that the current accountability system is perverse and has perverse outcomes and needs to be changed and made more balanced. We are struggling to get a great deal of expert input into what a more balanced accountability system would look like that we could recommend to Ministers.
Andrew Hall: I am sure Mark and I could have a competition as to who has the most, but between us-I am sure that Pearson have the capability as well-we have these people that we develop over time. That is why we have our international research committees. That is why we have our standards committees that we have had 10, 15 years.
Rod Bristow: We would love to help.
Damian Hinds: If invited.
Chair: If you could jointly or separately present us with a beautifully gift-wrapped solution, we would be inordinately grateful.
Q553 Charlotte Leslie: I just want to ask a quick question on something to do with data before I return to the public’s perception of and faith in the system. It is just a simple question: do you collect, and could the Committee see, data on what percentage and number of children on free school meals take your qualifications?
Andrew Hall: I honestly do not know the answer to that. I will find out and write to you.
Charlotte Leslie: If each of you were able to submit that to the Committee, it would be fantastic. Thank you very much.
Rod Bristow: I think we will all do that, yes.
Q554 Charlotte Leslie: Returning to faith in the system, I am afraid I am going to play devil’s advocate here in talking about conflicts of interest, which I think is one of the things that hits the headlines and the public are concerned about. In one of the submissions to the Select Committee, a situation was described-and this is a situation described in evidence, so I would be very interested to hear your response to it-where Edexcel apparently blocked an endorsed publisher from attending a series of events where Edexcel presented an Edexcelowned textbook by an individual who was alleged or said to be both a Pearson author, so producing materials, and also a chair of examiners for a subject, paid to develop the exams. Apparently, there was no other mention of endorsed textbooks, although an endorsed publisher had wanted to go to this event and was blocked. It was said that the individual’s textbook would be sent out with a specification and, from the evidence we have, the endorsed publisher thought there was no point in turning up to any more of these things. It feeds into this concept of conflict of interest, so I wonder just how you would respond to that.
Rod Bristow: I understand that. Just on that specific incident, I do recall hearing about it at the time. There was a mix-up and there were probably slightly two sides to the story as to what happened. I spoke to the chief executive of the company who made that submission and I just said at the time, "This is not our policy. This is a mix-up. If ever you hear about anything like this ever happening again, please let me know." It is not in our interests to discourage other publishers from producing resources that schools require for qualifications, and we have very clear policies on this. In fact, if you look on our website you will see that we promote the textbooks from other organisations and we have very clear policies about making sure that we have complete evenhandedness, including in the way the textbooks are endorsed. We make sure the process that is gone through applies equally to textbooks that come from other organisations as well as the ones that we produce. So we have very clear policies in place.
We also have very strong conflict of interest policies and firewalls to make sure that there is no chance of any knowledge of what is in an exam paper getting out to somebody who has a relationship with a school or a centre. There is no chance of anything that is going on in any publishing activity leaking back or influencing what is in the question papers.
In addition to that, we introduced last year a new code of conduct for our examiners, whereby we said to them, "We no longer want you to be the main authors on textbooks," because we feel-this is not just an issue for us-it is putting examiners in quite a difficult position.
We also last year made a change where we required our examiners to no longer be directly involved in the training of students for the same reason: that it was putting them into quite a difficult position. I think we can go further, and I think maybe if we do there is a way that we can do that collaboratively across the industry.
Q555 Charlotte Leslie: This is a question for all three of you: given the potential for certain biases in the system due to the financial incentive, with the best will in the world, do you think there is an argument to be made that the JCQ should be stronger and that selfregulation perhaps is not enough?
Mark Dawe: I am the chair of the JCQ and it is a membership organisation. We do a lot of work to have our code of practice and everything else, but it is a membership organisation of the awarding bodies. So I think it goes back to the role of the JCQ to try to find things that help all the schools to have common practice and the role of Ofqual as a regulator. I think we have to divide that very clearly.
On the textbook, I think this is the area where we are going to differ, and I do not support some of the things that go on in the market; you heard from OUP of a number of practices. There is advertising at the moment offering "come to our qualification and get our textbook half price as well". Personally, I do not think that is appropriate. We work, as OUP said, with a whole range of publishers. We tendered out every subject, or sometimes bundles, again to make sure that the minority subjects got support. We make no money from those publishers and anyone can come along and say, "I have something that we would like to help support your students." We take a look at it and say, "Yes, that would support the students," and that is when it gets the OCR badge. It is saying, "Yes, we have had a look at it and it seems to meet what the students would need." We are very comfortable with that process and think it is the way things should work.
Andrew Hall: I made clear at the beginning that is where I have certainly taken our organisation.
Rod Bristow: On that specific point, we do exactly the same thing. We make sure that no textbooks, including our own, will receive an Edexcel endorsement unless they have been through a very rigorous process. We are very evenhanded about that and we are completely nonexclusive in the way that we have publishing relationships. Indeed, we publish for other awarding bodies as well.
Q556 Charlotte Leslie: From the concerns that exist, do you think that if the JCQ is not the body, perhaps, to do it, Ofqual needs some more teeth or more focus in that respect?
Mark Dawe: I think Ofqual has the teeth; it is about focusing their work on the results, and I think you heard from the researchers the other week that that focus and expertise is not all in there at the moment. I go back to the point that standards and comparison should be their focus and then everything else falls out of that.
Andrew Hall: Agreed.
Rod Bristow: Yes, but I think we should be open to a discussion with Ofqual about this and welcome Ofqual’s involvement, and I think Ofqual is looking at this issue.
Q557 Ian Mearns: Mark, you mentioned a code of conduct; is that available? Can we have a look at it?
Mark Dawe: Yes, it is on the website.
Q558 Charlotte Leslie: Going back to perception, it is the old "dumbing down" argument. In a simple sentence, do you think kids are getting brighter or better at passing exams, given the external things that are not changing, like PISA and feedback from universities and employers?
Rod Bristow: I would say they are getting better taught. I think the evidence is that if you really concentrate on practising doing exams you will get better at passing them, so I think there is a degree of that, and that is one of the things we need to look at.
Andrew Hall: I think they are becoming more skilled at taking exams. There is a focus on that. There is better preparation than there ever was, certainly in my day. I think there is also a greater understanding of the importance of them across the whole of society. I think that is a change, and the teaching is improving.
Mark Dawe: I support what was said and I go back to my point about transparency. In the past, a select few had the inside track and understood what was necessary, and now we all strive to make sure that everyone has that opportunity and not just the select few.
Q559 Charlotte Leslie: Just briefly to go back to data again, can I ask what data each of you share with Ofqual about grading and whether it is publicly available and, if not, whether the Committee could see it?
Andrew Hall: For every exam series, we share the outcomes for our results, and we share them with each other, not just with Ofqual. There is a degree of peer challenge, it would be fair to say, on occasions-of understanding why. We work to a certain degree of tolerances, and if a particular award is outside that tolerance, if the grades have gone up or down by too much, we have to be able to justify that; it is shared. We certainly provide that data to Ofqual regularly and also, after the series, our technical people between Pearson, OCR and AQA carry out some statistical analysis, and I know we have shared that with Ofqual. Speaking personally, I would be very happy for that to be shared with the Committee.
Mark Dawe: A key role of JCQ, I would say, is the technical stuff, and the technicians work very independently from us, so there is no fear of using the data in an inappropriate way. They use it to look at the data pre and post the awarding series.
Q560 Damian Hinds: Why do you share that data with each other?
Andrew Hall: It is part of the maintenance of standards-to be sure. I do not want to repeat the long lecture or the information you were given by the technical people, but one of the ways of making sure that the grading standards are maintained is to use the board, and the expertise to look at the challenge rests within the board. As I said earlier, Ofqual is getting more and more capability in that area, which is to be encouraged, but I think it is something that has been helpful to be sure that the boards were each confident in the other.
Mark Dawe: We have done a factsheet for you, which I think you have received. Factsheet number two explains the maintaining of standards and that explains the sharing of the data and how the data are used.
Q561 Chair: Mark, you said that you were ensuring that the information about how best to tackle the examinations was shared with everybody and not just the select few, but the gap between the select few, namely those in selective state schools and the independent sector, and those at comprehensive schools is widening. It does not paint such a benign picture when you look at the actual data for A Level results, for instance.
Mark Dawe: As an educationalist, I would go back to the point that was made in the earlier session that a broad education covering the whole topic area and not focused down normally guarantees the best results. That is what we need to ensure happens in all our schools.
Q562 Ian Mearns: Can I just quickly go back to the code of conduct? Is that your own within your own organisation or is it shared across?
Mark Dawe: There are shared codes and then, obviously, each organisation has their own codes and policies as well, often drawing on those shared codes.
Q563 Ian Mearns: I think we are trying to tease this out: there is widespread concern and disquiet about the relationship between examiners, authorship, publishing and the examination process itself. But the way that you have described it, Rod, in terms of the firewalls that exist, sounds much more like a virtuous circle than a conflict of interest. How are we going to get over the widespread concern about these relationships?
Rod Bristow: I think it is doing the sorts of things I was talking about, but I think we should be considering going further than that. It is important that if we do that, we do it collectively, I would say, because otherwise there will be imbalances, potentially, between the awarding bodies in terms of how examiners are dealt with or how we work with examiners, and that could be a problem. That is the reason I said I wrote to my colleagues but I have also raised this directly with Glenys Stacey, because I think it is an area that we should discuss and I think we can make more progress in this area.
Q564 Ian Mearns: Kate earlier on gave us some examples of endorsements on the back of textbooks and examination course preparation papers. We really have to get over that, haven’t we?
Mark Dawe: Between 75% and 80% of our examiners are existing teachers; 20%odd are retired teachers. They are part of the system. We are all part of this education system and it is a shared issue for all of us to deal with. It goes back to a point that I think was made earlier. We need schools to say examining is important and we will support it. We are looking at providing examiner training, maybe using the teaching schools network as a way of encouraging people to look at examining, because it is not only external examining. They can take some of that back into the classroom in terms of assessment and what they have learnt about what good assessment is. So I think we have to raise this for the whole system, but I will emphasise the point: examiners are not our employees; they are the teachers. There are certain areas where I think we have all sat back, given what was in the Telegraph and other things, and said this whole system has relied on trust and, generally, they are very trustworthy people. But if that trust is evaporating, we have to put certain barriers in place, and it may be that we are reaching the point where anyone who has seen a question relating to the future cannot be involved in seminars or books, because they have that question in their head. We have trusted people, and whenever there has been a problem-and out of 13,000 examiners, you will get one or two-we have dealt with it rapidly, removed that examiner, so it is dealt with. But if the sacrifice we have to make is to put some of those things in place to regain the public’s trust, that is what we are going to have to do.
Q565 Ian Mearns: That level of trust you have talked about seems to have evaporated to a large extent, and therefore something does need to be addressed. In previous sessions with examiners themselves, they have talked to us about the training courses that teachers can go on run by the exam people, where 200 teachers at a time go through a process leading up to the exam. Is it correct to say that textbook endorsements and training courses are not significant income streams in their own right, but that you have to offer them to have a competitive offering in the exams system itself?
Andrew Hall: Can I talk about textbooks on that? Having looked at this for some while, I know what my real concern is. Rod’s organisation publish some perfectly good textbooks, some very good textbooks-
Rod Bristow: Thank you.
Andrew Hall: -that cover specifications we offer. Anybody can publish a textbook, and we have been talking to the teachers in schools over an extended period saying, "What do you actually want? What comfort do you want?" What they want to be sure of, if they are going to use a particular textbook-and I value choice, because different styles suit different teachers and suit different schools-is that they know that if they study the syllabus as outlined in this particular book, it is sufficient to cover the syllabus.
The worry is if we do not, as OCR do and the way we are now wanting to go, provide that level of comfort to teachers-that we, as an organisation, have looked at this particular textbook and we are satisfied that it is fit for purpose-we leave particularly the newer, less experienced teachers exposed. That is a clear message we have had back. The more senior, experienced teachers will use generic textbooks. They might use an OCR textbook to study an Edexcel exam, because they have that level of confidence. So it is finding a way to protect that.
Some of the things printed on the back of the books make you just want to find a hole, crawl in and die, because they are horrible, and we do need to stop that.
Mark Dawe: With seminars, it feels strange that as an education and training community we are saying we are not going to educate and train our teachers. There should be support. My original job was an accountant, and when I joined a college I was trained in how to apply my accountancy skills to a college environment. Our training is about taking teachers and their broad teaching skills and explaining how the qualification relates to their teaching, and we get issues, even with the seminars, if there is someone who has taught the wrong book. We have spelling, punctuation and grammar coming in, terminal assessment in GCSEs. Are we saying we just leave the teachers to find out themselves, read a couple of things on the website and have a go at it, or are we saying we should support those teachers and give them something?
There is another important thing. I sat in an awarding meeting yesterday in English. The principal examiners interact with those teachers, and they are picking up messages about "this does not make sense" or "I do not like this", and it goes in and that is part of the development of the whole system again, and we should not lose that interaction.
Q566 Ian Mearns: Why would examiners, who are also teachers, express those concerns about the sort of seminars that you have talked about?
Mark Dawe: I do not know. Clearly, the Telegraph identified some inappropriate wording and things that were wrong, but underlying it there was a lot made available. It goes back to a point I was making earlier about the transparency of the system, and if people do not like that, we need a proper debate and to make a decision about where that line is drawn. But it is a system that we are required to follow at the moment, and we do follow and we make it as transparent as we are allowed to, because that is what we are being asked to do.
Rod Bristow: The important point is that debate about where the line is drawn really should be had. I think it is a really important issue.
Q567 Ian Mearns: Do you think it is time for the selfregulation within the sector to be looked at by Ofqual in terms of a much more robust analysis of exactly what is going on, so that everybody out there can be confident that we have a clean and upfront system?
Mark Dawe: I think we are driven by the regulator already. It is getting the regulation right. We are suffering at the moment from a situation where the regulator is meant to be regulating the awarding body with 161 criteria that we have to meet by 18 May, or at least say how we are going to meet them, and a system which is regulating individual qualifications as well. So we are being doubly regulated at the moment and, as the researchers said, there is not a large resource in Ofqual. It is about focusing it on the right thing, and I come back again to standards and the comparison work being the thing. If they focused on that and got it right, it would give the public the confidence they are looking for and that would help us as well.
Q568 Ian Mearns: The bottom line, again, is about public confidence and trust in the system. Therefore, from that perspective, if that is undermined to the extent that we believe it might be, you may need to do something about it.
Mark Dawe: Absolutely, yes.
Q569 Charlotte Leslie: Just quickly going back to the JCQ, given the roles and responsibilities and remit that it holds, do you think it is suitably accountable to the public? If you could just briefly-probably this is mostly directed at Mark-describe some of the activities, roles and remit of the JCQ, that would be very helpful.
Mark Dawe: The main function of the JCQ is to ensure that there is consistent practice across us all when it comes to general qualifications examining and that we follow a common timetable, and we make it as easy as possible for schools to run their exams in an operational sense, so we have some coherence between us so that they are not having to look at four or five different ways exam boards work. Then there is the technical work we have already talked about, ensuring that there is proper sharing of data so we can ensure across the boards that standards are maintained. I would say those are the two main functions.
If there is a common issue that has come up, we will look at it, but we are also very, very aware that we do not want to be seen as acting as a cartel. So there are certain things at the meeting where the chair will say, "No, we are not going to discuss this, because it is inappropriate."
Q570 Damian Hinds: Do you have lawyers present when you have a meeting of the JCQ as an American organisation would?
Mark Dawe: No.
Q571 Damian Hinds: At lots of operations in America, you would never have competitors meeting in a room without some greater guarantee than just the chair’s discretion for nothing of an anticompetitive nature being discussed, would you?
Mark Dawe: Where there is a concern, we stop, and where there is a hint of a concern we might turn to Ofqual and say, "We do not think, as JCQ, we can discuss this as a group, because it is not appropriate for us to be discussing." But if you come back and say, "Actually, we want to have this debate," about examiners or whatever else, then the regulator is then saying, "Yes, this is something that we should be talking about."
Andrew Hall: Just to be clear, I think there is a danger of the JCQ’s importance being overstated. It is very much about easing the administration of exams-dealing with students who have learning disabilities who need special access-to make sure we have common arrangements for that. It is about making sure we have timetabling, so we do not all set subjects that clash on the same day. It does not control, I do not think, with my perspective, the activities of the awarding bodies.
Rod Bristow: But I think there is a fair question about whether it should get much more involved in addressing issues around standards and having those discussions around standards while, at the same time, making sure that it is not straying into issues that are anticompetitive.
Q572 Damian Hinds: I suppose it does seem a tiny bit anomalous, because I think a man from Mars hearing this discussion would assume that a lot of these things were the role of Ofqual, not the JCQ. I am sure they are lovely people. I have no reason to distrust the JCQ at all, but it just seems odd that as far as we can make out, and from the JCQ’s letter, it is not regulated as a body. The members of it are, but it itself is not. To give one example, Andrew, I think you mentioned this point about special access for children with special needs, which is a very important area, but there is a matter of public interest in how that is done and data from the Department for Education say there has been an enormous increase in the amount of extra time awarded, for example, in the exams. There could be perfectly good reasons for that, but it seems to have happened at a time when there has not been a massive increase in the number of statemented children and so on. So to whom is JCQ accountable for those arrangements?
Andrew Hall: I am going to answer this as far as I am able, and someone else may need to help me. JCQ has just taken over the administration of large chunks of the system from QCDA on QCDA closing, because it had to go somewhere. It was a centrally provided thing and we stepped in and said, "Well, it is right that there is commonality." So that is about the administration of the system. I, truthfully, sit here not knowing who decides how much the extra time should be. As embarrassing as that may be, I do not know. If it happens in my organisation, it will be handled by a specialist team, because they are very challenging areas.
Q573 Damian Hinds: I think we understand from the JCQ letter-I am looking around me for confirmation-that decision had been devolved to individual examination centres. Again, there could be perfectly good reasons-I am not doubting that for a moment-but that in itself is a matter potentially of public interest, so even if that decision has been devolved, to whom is the decision to devolve accountable?
Mark Dawe: It goes back to this common operational system that we all operate and then we all operate in our individual organisations as regulated organisations. If Ofqual turned round and said, "We do not like how this is being done," then as JCQ we would look at changing the common operation and then us all changing together, rather than one having one system, one having another. So it is just an opportunity where there should be no differences between us-that we can have that common system and all apply it, and if someone thinks it is not right, we, as a group, can look at it and change it without falling into the competitive issues.
Q574 Chair: You all want Ofqual to be stronger, so on that basis I assume you welcomed its power to fine you. You are nodding.
Rod Bristow: Yes.
Mark Dawe: I am not sure it is the best power to have in this market. It is one of many powers and, at the moment, we are all talking to Ofqual about undertakings we are making in relation to the exam areas. That is a power that is asking us, "What are you going to do about it?" and, basically, there is a timetable and things that we will be doing and a lot of money we will be spending. To me, that is what the focus should be and getting us to spend our money on the things that will make a difference. I am just not convinced, especially where it is predominantly not-for-profit organisations, that fining is the answer in this case. They can have the power, but I think some of the others powers are far more appropriate.
Andrew Hall: In my view, it is fine to have the power, but I am not sure it is the most appropriate sanction for everything and it needs to be used carefully, but you may as well put the power there. I go back to the point, and I kind of agree with Mark on this, that getting a real focus and building up the ability and the capability so there are a number of people that can really engage in the standards issues there is important, and there have been real improvements over recent months. The standards board is a major, major step forward, because that is bringing advice in.
Q575 Chair: Ofqual is consulting on how it should use its additional powers. Just to finish off, can you give us a quick summary of how you think Ofqual should behave in order to ensure that you are focused on what is most important and not focused on things that are not?
Mark Dawe: We have said also with the fining there should be a commissioner there, because there is going to be a big debate about what the appropriateness of a fine for a particular issue is. We believe that if they are about regulating awarding bodies, let us make it regulating awarding bodies and not all the qualifications as well. Pick one, but then stick with it, and I go right back to the point that they need to be really robust about the standards, and if they have proper researchers in there and they do the research and say, "We are not happy with this qualification," then action is taken quickly.
Q576 Chair: Haven’t you just said you do not want them to regulate qualifications? You said they should just regulate you guys and if they do that well enough-
Mark Dawe: But standard of qualifications is slightly different. In terms of regulating qualifications at the moment, we have to meet all these criteria to get a qualification approved and tick all the boxes. Standards are a different thing when reviewing across us and across different qualifications. Are they up to the standards expected and doing something about it if it is not felt they are?
Rod Bristow: I think for the purposes of public confidence in the system, the public would expect that the regulator would have the ability to fine. For that reason, given that public confidence is so important, I think it is right, and we wrote to the Schools Minister saying that at the time. It is important though, of course, that those powers are applied judiciously, appropriately and all of that, and that they apply to the relevant proportion of income of an awarding organisation, but those are the sorts of things we are in conversation with Ofqual about.
Q577 Chair: Moving away from the power to fine to the use of additional powers, how should Ofqual best use them to raise standards rather than become a bureaucratic irritant?
Andrew Hall: It is about where their focus of effort and attention goes. I think Alison Wolf in your earlier session with the technical people suggested that some of the work around economic regulation was of limited value; I am probably paraphrasing it, I suspect. I think it is about making sure that focus is there. To my knowledge, there have been three studies that Ofqual in its various forms has carried out into pricing and how the market works-we touched on that earlier-none of which reached any earthshattering conclusions. I think it is about where the resource is supplied and it is, therefore, I think, about the content of the qualifications: what do the subject criteria look like, are they complied with, and running comparability studies across the board, because the devil in this is in the detail. It is not in the high level, so it is running really indepth comparability studies that probably cost about £100,000 to do, but once done provide real evidence about where a standard is. It would enable some of the questions that over 10, 15 years are hard to answer to really be tackled and, taking Rod’s point, to be sure we are in the right place.
Rod Bristow: Glenys Stacey has asked a very good question, which is: what are the incentives that are in the system that drive standards up? I think that is the question that we need to be engaging on and that is where Ofqual could play a role in coming up with some proposals-and we would support them on this-to see what incentives can be put into the system that would have an impact on standards. So, things like the degree to which the comparability data between awarding organisations show what they should show and, if there are gaps, what would be the consequences of that. There are other ideas like that.
Mark Dawe: It is our job to maintain and raise standards, and it is Ofqual’s job to make sure that is happening.
Chair: Gentlemen, thank you very much for giving evidence to us this morning.