Examination of Witnesses (Questions 20-39)|
WEDNESDAY 15 MAY 2002
20. But in looking at yourselves you are saying
that there are no major weaknesses in the QCA as currently constituted?
(Sir William Stubbs) Absolutely correct. I see no
major weaknesses. There are areas where no doubt we can improve
and we will be anxious to improve when we see the evidence of
it, and in some areas we have identified them ourselves and we
have improved, but not major weaknesses, Chairman.
(Ms Evans) I just wanted to add that although there
is not a formal part of the Quinquennial Review process that invites
us to self-assess in the way that you are describing, the process
is interested to see what the organisation has achieved over a
five year period as well as to try and identify if there are areas
where we are not achieving and we are not organising ourselves
well. From what I know about the way in which the process has
been conducted, I feel confident that the report of the process
will list our achievements, will discuss what we have achieved
as an organisation in addition to making recommendations for change.
21. And you have revealed that the QCA conducts
its own evaluation. Within the last annual self-evaluation what
were the issues that you identified as those most urgently needing
(Ms Evans) When you indicated earlier that some bodies
who had written to the Committee had been critical of the QCA,
we do have a role that is sometimes termed the regulator of awarding
bodies and very often, if you were to ask the regulated how they
feel about the regulator, you are likely to get comments that
would be critical of the regulator because that is the nature
of the way that that works. I do not think we would want to have
a cosy relationship with the awarding bodies and I am sure you
would not want us to have that role. If there are signs that you
are picking up from bodies of that sort, that they are not particularly
happy with the way the QCA goes about its business, we would be
very interested to hear of any specific examples, if you were
willing to share those with us. It could be for that very reason.
22. I appreciate that but that does not answer
my question. My question is, in your most recent annual self-evaluation
what are the issues that you as an organisation identify as most
urgently needing attention?
(Ms Evans) We have some large executive activities
in addition to the regulatory role. One of those activities is
in relation to assessment tests which the Chairman has been talking
about. That is a very large scale process that the QCA conducts
every year. You will have spotted from press reports recently
that sometimes these things go wrong, so a high priority for us
in reviewing our performance is to examine whether there are any
improvements that we could make in that process. That is a priority
for us at the moment. I would say that another area is in relation
to the way we work with awarding bodies, particularly with the
unitary awarding bodies, through bodies that are responsible for
GCSEs and GCE `A' levels. That is an area that has had a great
deal of change with the introduction of the Government's reforms
of `A' levels recently, so that is where we are spending a lot
of our time, focusing on those arrangements to make sure they
are being conducted properly.
23. Your history is of an organisation emerging
from a variety of predecessor organisations originating with the
Schools Council. Do you think the Schools Council ethos or dominance
of the schools work in the organisation is restricting what you
are able to do with the adult population? In your introduction,
Sir William, you mentioned the importance of adult schools. What
is the balance of resource and staffing within the QCA between
work on schools and work on adults?
(Sir William Stubbs) As it so happens, Chairman, we
have a former member of the staff of the Schools Council here
who is now a senior colleague responsible for qualifications,
including vocational qualifications. I would say to you in general,
and I will ask Keith to follow it up, that for those who read
the daily press, the weekly press and so forth, it would be very
easy to come to the conclusion that we are preoccupied with schools,
but that is how the press choose to treat vocational qualifications.
That is the way they are concerned. There is no great excitement
about NVQs and there are some great success stories to be told
about NVQs. The progress that has been made on vocational qualifications
over the last five years is a credit to a lot of people in business
and industry and commerce and many of Keith's colleagues that
the system is much more reliable than it ever was, but it does
not make the front page for reasons we understand but may not
agree with. That is the backdrop and I can assure you that there
is much effort that goes into this. As to the specific question
about staff, Keith will maybe wish to hark back to Schools Council
(Mr Weller) I will not go back to the Schools Council;
I will spare you that. I think it would be fair to say that there
is a profile of expertise on the staff that is properly reflective
of the range of tasks we have to do. There are very large numbers
of colleagues who work with me on qualifications and in our quality
audit division, who have their background in awarding bodies or
in training organisations or in further education or in adult
education or in industry. There is a fair spectrum of that, and
of course we have people with schools expertise as well and local
authority expertise too. It is a pretty wide profile of expertise.
I could not pretend that we had all the expertise we need in-house.
We could not possibly do that. What that means is that we must
work with consultants and collaborate with those people who do
know employment sectors, who do know their subjects and ensure
that we take those things on board when we make the judgements
that we have to make about the need for qualifications or, if
required, any gaps that there are, and the quality of those qualifications,
and we do that widely. We are in deep conversation with specialists
outside the organisation more than we are with specialists inside
the organisation. There is one thing I want to point to that we
need to continue to work at and that is our links and connections
with the world of employment. It is quite difficult to get employers
to commit time on a regular basis to do work for an organisation
like us. We try regularly. We have employers on the board itself
and on our advisory committees. We have ex-employers on the staff
and we have very close links with the training organisations and
with the employers' associations. One of the first things we did
when we were created was to set up a number of sector advisory
groups that bring together the key employers, large employers,
and training organisations in those sectors. The object of that
was to give us authoritative advice on the range of qualifications
that were needed for the sector, but it was always our intention
while drawing on that advice and having all those people meet
over the course of a couple of years in order to map the territory
that we should hand that back to the sector itself. It is not
for us to keep those as standing committees and in many cases
those committees have now been replaced by a network of employers
and training organisations outside.
24. In respect of the in-house staff could you
give us a figure in terms of the approximate balance of in-house
staff working on schools matters and working on other matters?
(Mr Weller) Yes, I think I could do that. We have
something of the order of 60 or 70 officers who are the specialists
in the employment sectors and the school subjects. Of those about
half to 60 per cent are working on specifically vocational areas
and about half of the others have subject areas where there is
both a vocational and an academic dimension, for example, foreign
languages. We have a single team that works across the range of
academic and vocational qualifications. It might interest the
Committee to know, and I do not think last year was absolutely
typical, but a huge amount of our resource of time spent by staff
and financial resources was devoted to vocational qualifications,
partly developing the occupational standards that qualifications
are based on, but also developing a whole range of technical certificates
using modern apprenticeships. I would have thought there was a
ratio of nine to one of our financial resources devoted to vocational
work last year.
25. We have heard about how the QCA has come
together from a number of bodies and the history of the organisation.
Often nowadays we live in a world of rapid change and sometimes
change is associated with an idea that change must mean progress
forward. I would like to ask two questions around this issue.
First, how do you feel the role of the QCA has developed since
it was established?
(Sir William Stubbs) The role generally?
(Sir William Stubbs) The great strength now is that
the procedures, the thinking, the discipline that surrounds having
in place rigorous qualifications with respect to, say, general
education in school subjects, is now matched, as you have heard,
in applying that to vocational qualifications. There is an integrity
now about the national framework that was not there before. Number
two is that the knowledge and understanding and the experience
of the development of the school curriculum and the specialists
that we have there informs, now directly, people down the corridor
who are engaged on the assessment of that curriculum in schools.
That is an integrity that was not there before. The organisation
has done something that its predecessors just could not do so
easily. It was not easy at the beginning or in different buildings.
Now we are in one building and that is real progress. There is
a comprehensive link running through the organisation from which
the various parts draw strength.
27. Given that we know that some of your predecessors
have quite short life spans and that, whether they are justified
or not, there are a lot of criticisms around and that people will
always want to say that something could be better, what is your
strategy for ensuring that you do continue and that you do continue
to develop into the future if you see yourselves as a positive
organisation which is doing a good job?
(Sir William Stubbs) There is no point in continuing
into the future for the purposes of continuing into the future.
The purpose is when you can draw on what you are doing and you
are convinced that there is a good and sound contribution that
you have made to the quality of life. I am in no doubt from watching
what has been happening, and I have been Chairman since the start
of the organisation, that the contributions that the QCA has made
to the quality of education in schools, colleges and the aspects
of vocational and adult life we are concerned with have been quite
distinct, and there is now about our national arrangements a confidence
and a quality and a rigour that was not there before. We presented
that as evidence to the Quinquennial Committee and it is a matter
that the Secretary of State, I am sure, has in her mind from time
to time when she is looking at our work.
(Ms Evans) We have a strategy in terms of our internal
policies in QCA to try and recruit the most able people that we
can right across the range of our activities. The quality of our
advice to ministers is going to be a very high priority for ministers
in determining whether they want a body like QCA to advise them,
and so the quality of the work that we do is important to us.
I think the strategies we have in place for recruitment, for coaching
and training of staff, for trying to ensure, thinking about your
colleague's question earlier, are ways in which we can allow the
very different range of people whom we employ in QCA to be working
together, so that we bring the coherence, which was part of the
original idea in setting QCA up and the reality of that into the
way in which we are advising ministers. It is the quality of what
we are doing that we are judged on.
28. You are presenting a very positive picture
there and defending your organisation, which I would expect you
to do, but how undermined are these important aspects of your
work by such things as the school test papers not being delivered
on time, those kinds of administrative issues which are not being
delivered in the way they should be? Obviously they are causing
problems for a whole range of people and have quite a high profile,
quite rightly, in the press.
(Ms Evans) I have set out the positive things that
we want to do. I would not want to say that those are not very
challenging things to do, particularly the sharing of knowledge
and experience across the organisation, which is very challenging
for us, so I would not want you to think that we are complacent
about the way we are moving forward in that area. We aim in areas
like the assessment tests to ensure that all the test papers that
we are responsible for developing and dispatching are there on
time in the right order and in the right place at the right school,
but clearly that does not always happen. That disappoints us greatly
and that is why I said to your colleague earlier that we want
to ensure that the systems that we have in place for that part
of our business are as sound as we can make them.
29. "As sound as we can make them"does
that mean you think you can do things on time or that you think
it is always going to be a 98 per cent or 99 per cent achievable
(Sir William Stubbs) We cannot reduce the levels of
probability formally to zero. Given the scale of the enterprise
that is just not practically achievable, but what we have in place
is a range of techniques that enable us to be assured that young
people's interests are well protected and there are contingency
arrangements in place at various stages but, sadly, there are
occasions when there is a breakdown. The press report you mentioned
seems to be one. That is regrettable and I apologise; it should
not happen, but the test of a good organisation is, has it continuously
held up to some extent and ameliorated the damage? If the case
you are referring to is the case I have seen, the tests got to
the school late and the tests could still be carried out within
the school week. Also, the test of a good organisation is, can
it learn from these errors and then next year make it even more
secure? I think we can give you that assurance, Chairman. You
can well imagine that there are lots of people, probably some
sitting behind me now, who are looking carefully at our procedures
and would not hesitate to bring public scrutiny, quite properly,
to bear if there were breakdowns. I think the number of breakdowns
that you actually see are very small indeed.
30. All I am suggesting is that to a lay person
it does not seem that difficultand I might be completely
wrong hereto get some examination papers from one place
to another on time. We know the timescales in there. What I am
suggesting is that a failure in something which on the face of
it seems to be relatively simple undermines all the other good
things you are talking about on integrating, regulation, assessment
and all that kind of thing. Surely you can do a bit better.
(Sir William Stubbs) If we could ensure that 16 million
scriptsand that is what we are talking aboutmove
around the country over the course of eight weeks, if we could
guarantee that there would not be one go astray, Chairman, you
can rest assured that by now we would be doing it. What I am saying
to you, and I am choosing my language carefully, is that we have
in place a range of procedures but we cannot reduce the level
of probability to zero but I think we have done everything reasonably
possible to minimise it. As I say, when something goes wrong an
organisation must move promptly and sensitively to deal with it.
It is when they meet a blank wall that it is worrying. I assure
you that they do not but we still want to investigate.
(Ms Evans) I can understand that it seems a very simple
process and I cannot but agree with you that it is a straightforward
process. The way in which we incur risk in this process is because
of the scale involved. In fact it is 18 million scripts that are
delivered to schools.
31. You did not even get that right.
(Ms Evans) It is the size of the exercise that means
that we have to have as good a process as we can get. It is the
size which introduces the difficulty for us because we are aiming
to get something into every single school in that way over a very
short period. We do not want mistakes to occur and we are looking
to learn from everything that has happened this year so that we
can try and do better next time.
32. I want to move on to examination boards
but, before I move to that general area, one of the problems we
still have is not in terms of delivery because it is a real competence
you have of delivering 18 million scripts or recruiting excellent
staff. We have great admiration for that because you have not
been in front of this committee before. The fact of the matter
is, what concerns people outside, and we have to reflect that,
is that there seems to be a bit of a cosy relationship between
QCA, which is about standards and the curriculum and all that,
and the department. However brilliant Beverley Evans is, she is
a civil servant. She is your Deputy Chief Executive, seconded
from the Department of Education and Skills and in a sense that
is symptomatic of a very close relationship. What people out there
say to us, whether they are academics or professors or ordinary
constituents, is that for a standards body is it not a bit cosy
that the relationship between the department and your body is
not as obviously independent as, say, Ofsted? What would you say
(Sir William Stubbs) I would say that the word "cosy"
is a pejorative one. The phrase "close and understanding"
is rather different. We do have a close relationship. There are
aspects of departmental policy that we would like to discuss and
influence in the early stages and there are aspects of our organisation
and the things we do that the department wish to understand in
order that the Secretary of State can be briefed. However, civil
servants do not encroach into areas that would be improper; there
is no question of that. If lying behind your question, or the
remarks that have been put to you, is the idea that in some way
the political will, or the departmental executive, runs in determining
standards or levels of tests, and therefore it is a cosy relationship
that way, I utterly refute it, Chairman. That would be, I think,
a serious flaw in the arrangement.
33. That is my job and our Committee's job,
to draw out this feeling that that if it was too cosy or too close,
whichever way you have described it, here is a Government that
says, "We are driving standards up. Aren't we good? You can
re-elect us because we have increased literacy and numeracy, standards
at 16", at the same timeand this is an insidious view
perhapsthat they might be bringing pressure on you to relax
the standards in order to make the figures look better.
(Sir William Stubbs) Chairman, if there was any political
interference with standards of examinations I would resign, but
I have never seen an inkling that the two Secretaries of State
I have worked with and their ministers have in any way sought
to encroach into that territory. They have been very careful,
and very wisely very careful, to keep away from that. There has
to be an integrity about standards that can be seen. If you say
we could be more overt, then I will listen to that and we can
be more transparent, but the arrangements by which standards are
set are open and transparent and the books are there to be seen
and are seen, and I think you can draw confidence from that.
34. There is a little bit of growing concern
about the reliability of examination boards, the three boards
that serve England anyway. It is not just Edexcel. We know the
three examination boards themselves are short of teachers to mark
papers and so forth. I am slightly playing Devil's advocate here
but is not the answer that we have a single examination board
in order that we have a more effective and efficient service which
can benefit candidates and teachers alike?
(Sir William Stubbs) I do not think that a single
awarding body necessarily guarantees sufficiency and effectiveness.
We are well aware of a system not too far away from us where a
single body ran into difficulties and to some extent I think the
difficulty was because it was a single body because the computing
system for that body failed or came close to failure. This is
something that Sir Ron Dearing looked at when he was looking at
16-19 arrangements a few years ago. He was not persuaded that
there were benefits from a single body, that there would be a
net gain and that there were benefits from having more than one.
At that time there were more than three and he recommended that
they be reduced. They have been reduced to three. I think there
is a protection in that. It is important to understand that there
was a time in the past, Chairman, and we are talking about school
qualifications again but that is where the media attention is
so forgive me on that, when schools tended to link with one body.
That was the body in their region or it was a body they had worked
with for a long time. That has now gone and most schools deal
with most unitary awarding bodies because they want the choice
of subjects, the choice of examinations and so on. If there was
a partial failure of one body it does not ripple through to the
others and the school can continue to operate. That is a partial
protection and I think if people want to go down the path of having
one national body for school qualifications it would take some
very careful preparation and certainly the transition from here
to there would take very careful scrutiny and very careful management.
It would be fraught with some risk.
35. Accepting that broadly speaking you keep
the three examination boards, what is the QCA going to do to try
and instill greater confidence and instill greater awareness of
the good work that examination boards do? There are plans to increase
your powers, are there not, so that you can intervene directly
in all exam boards to correct faults that are there without having
to wait to be asked for an inquiry by the minister? Is that something
which you think is a good thing?
(Sir William Stubbs) I believe that is a good thing.
Can I just point out that this discussion is taking place in the
midst of a change in `A' levels which have been around now forit
was their golden jubilee of `A' levels last year, Chairman, although
it was not on all the front pages. They have been around for 50
years but the change that is taking place last year and this year
is the biggest ever. It is a huge change moving to modular examinations
and with the interim qualification. That has undoubtedly placed
stress on the awarding bodies. I pay credit to the way in which
they have attempted to deal with that and in the main they did
deal with it successfully. You have drawn attention to one where
the strain became too great and it looked like it was going to
fail, and indeed did fail partially. At that stage we were engaged
in an audit of it and therefore my colleagues were involved in
a close scrutiny of it and they saw signs that it could deteriorate.
It then did. All we could do at that stage was advise. We had
to be very careful that we did not get ourselves into the position
of a shadow director in the commercial world where we had given
a direction and then could be culpable and liable in the event
that the organisation as a consequence traded at a loss and went
bankrupt. That was a frustration. We could see things that needed
to be done and we did advise on them. Sometimes they would take
the advice, sometimes they were slow in taking the advice. I expressed
concern to the Secretary of State that we should be in a stronger
position when we detected a system failing like that so that we
could intervene and direct, and that that then would give more
security with regard to delivery of the examinations. After some
discussion and after much scrutiny, I am pleased that she is seeking
to put that in place in the education programme.
36. Do you think that will be enough because
there is this growing awareness that one or two of the examination
bodies are severely struggling? Edexcel, to be fair to them, had
4 million marks in 2000 and it will be 10 million this year, which
is quite a jump. We know that others are having trouble, including
teachers. Is that new power that you have been given by the Secretary
of State enough to re-instil confidence in the system?
(Sir William Stubbs) Chairman, I see no reason for
additional powers. Those powers will deal with the problems that
we anticipate could arise if things went badly awry. I can say
to you, however, that, as you would expect, we have been very
close to Edexcel and have been meeting with senior colleagues
and Edexcel since the turn of the year, and matters have improved
considerably. They have spent much time and effort. Part of the
difficulty in Edexcel was that the management became isolated,
one part from another; and therefore knowledge and understanding
of where things were going wrong, which is over here, was not
transmitted to another part of the organisation. They have worked
hard to change that and to improve their external relations. I
am more sanguine that this summer will be very much quieter than
the previous year.
Chairman: That is very reassuring. Paul, are
37. In some areas. In relation to the argument
for a single examination board, you were saying that the fact
that there were three allows a fair degree of choice between syllabus
and styles, but when I was first head of department in a school
in 1984, there were 12 exam boards and I could choose GCSE A-Levels
and some were radically different from others. All that has been
squeezed out by the National Curriculum et cetera. Is it
not spurious to say that three exam boards allow choice? It is
not the sort of choice we had back in the 1980s.
(Sir William Stubbs) I would not say the National
Curriculum resulted in any reduction in the number of A-level
awards on offer. That came about through one of our earliest tasks,
where the Minister asked us to seek to determine the extent to
which we could rationalise and reduce the number of qualifications.
It was not just A-levels, but there were something like 17,000
qualifications alleged to be around at that time. We have reduced
the number of A-levels and therefore the choice is less. It has
come down by about 60 per cent, but there are now 186 different
offers available to schools. The largest proportion is in modern
languages. If we derive it down to subjects, there are 86 different
subjects and 50, if you exclude modern languages. I would say
to you, Chairman: does a school system need more than 186 different
choices in exams? There is a body of opinion in many universities
that would say that early specialisation is not in the interests
of young people and they should keep it as broad as possible.
Indeed, part of the reform that is taking place in A-levels now
is to enable greater freedom. There is plenty of choice there.
In fairness to your remark, if we thought there was demand in
schools for an additional qualification, and it developed over
time, then we would want to respond to that and be sensitive;
but I think that most schools now are well satisfied with what
(Mr Weller) A multiplicity of syllabus and specifications
does not necessarily mean there is a vast array of choice. The
position a few years ago was that many of those specs were very
similar indeed, and it gave a massive job to teachers in schools
to work their way through them all to discover which one might
meet their needs. Part of the process in accrediting recently
new GCSEs and A-levels was to ensure that those specifications
that did come through in a more rational system were distinctive
and did offer a real choice. While there was a reduction in the
A-level range of specifications by something of the order of 40
per cent, I do not think there was a significant reduction of
real distinctive choice which made it easier for schools and colleges
to pick their specification.
38. I have to differ with you on that one. I
was head of the history department and when the AS-level system
came in, in my opinion the best courses were scrapped completely.
The point I was really making was that since we have moved from
12 exam boards to three, and most of the choice has gone, why
not go to one exam board and avoid all the complications?
(Sir William Stubbs) There is a case, Chairman, that
can be made in favour of one examining body. All I would say to
you is, if that was the decision and where the country wanted
to place itself, the transition from here to there would be exceedingly
complicated and would need to be handled with much care. Therefore,
you would need to satisfy yourself that the risks involved in
that journey were worth the net gains that would come out. I am
not saying you cannot work out a unitary, unified national examination
system that covers everything, but it is not a straightforward
matter to put in place and you will be well advised to think about
it very carefully.
39. The second question is that of the areas,
with examples. As we get older, our memories play tricks on us.
I cannot recall in the first ten to fifteen years when I was teaching
an instance where exam papers did not turn up, pages were blank,
questions were set that were impossible to answer. I cannot remember
that ever happening. Is that my memory being selective, or are
there more errors than there used to be back in the 1980s?
(Sir William Stubbs) The cases that you have quoted
are very much recent ones and very much associated with one examining
body. They were regrettable, and intervention has taken place.
I sincerely hope that they will not recur. As to whether there
are more now than then, there are more papers now than then: we
have modular papers, so you have six times as many. It is a very
hard question to answer. Where there is strain on awarding bodies
is not, I do not think, on construction of the tests and the printing
and circulation of those because they have worked very hard to
deal with that; the pressure is on markers. You have to get sufficient
markers to handle the choice. There is no getting away from that,
Chairman. They have struggled very hard. It remains a difficulty
in two subjects. I do not think you will have guessed those subjects,
but they are English and religious studies, where the A-level
qualification has grown massively over the last two years, and
they find difficulty in getting markers for that. They do their
best but, undoubtedly, it is a very demanding task for them. That
is what they are under stress for.