Select Committee on Education and Skills Appendices to the Minutes of Evidence



Annex C

  1.  This minute provides an analysis and summary of Peter Tymms' and Caroline Fitz-Gibbon's article "Standards, achievement and educational performance: a cause for celebration?", published in "Education, Reform and the State: Twenty-five years of politics, policy and practice", London: Routledge Falmer (2001).

  2.  In broad terms, this paper is questioning whether, in an absolute sense, the standards by which schools are judged have been maintained at a fixed level over an extended period. The authors' opinion is that, over the course of the last several years, standards have not been maintained, with the result that perceived achievement has risen. In other words, the fact that the attainment of pupils in primary and secondary schools is rising, marks obtained within the national testing framework are not a reliable indicator of whether they are actually any better at the subjects in question.

  3.  The political dimension is also discussed briefly. The authors remark that a drive to improve standards was the original impetus to extend the old GCE/CSE system to include testing at the four key stages and a unification of the public examinations at 16, with O-level and CSE assessments combined to give the current GCSE. Given the extent of these reforms and the large amount of assessment data that have been generated as a consequence, the question is posed as to what extent educational standards have actually risen, rather than perhaps appeared to do so.

  4.  Although the work of OFSTED is not in fact discussed here, the authors have produced other publications and given presentations in which they use exactly the same points to form the basis of a critique of OFSTED's work. For example, they have claimed elsewhere that inspections are excessively dependent on test results, but that there is inadequate statistical analysis of these and no attempt has been made to maintain standards. They thus believe the inspection process to be fundamentally flawed, since it is based on an unreliable system of assessment. It is worth pointing out that this opinion is contrary to their alternative view that inspections are invalid because of the OFSTED system of professional judgements; by making unsound test results the main culprit they are implicitly saying that JRS scores are not a major problem.

  5.  A brief study of Key Stage 2 attainment is presented, in which it is noted that attainment has risen significantly in all three core subjects over the period 1995-99. The data are that for the percentage of pupils attaining at least level four. It is possible that other measures do not produce such a stark contrast, but this would need to be investigated in some detail. There is, unfortunately, use of rather leading language in this discussion, making it difficult to believe the authors' view is entirely objective. For example, "The rise seemed to be levelling off Y but a sudden rise was observed the following year" is used to describe a feature on the graph that consists of only a single point. Similarly, mathematics is described as experiencing "a significant boost". Whether it is "significant" is a matter of opinion and "boost" implies action of an outside force, rather than improvement coming from the pupils themselves.

  6.  Comparison of data for reading scores of 11-year olds is discussed. The KS2 English results show a steady nine point rise over the latter half of last decade, whilst the other studies remain within five points of the normalised benchmark of 100 used in this data. The longest-period (1975-1995) study is American, and therefore may not be appropriate for comparison purposes, and the only study carried out over the same period as the KS2 results was by the authors at their CEM.

  7.  The remainder of the paper is given over to an investigation of factors that may have led to a fall in standards. One of these, which has been a more general cause for concern, is that teachers are "teaching to the test", so all we are seeing is an increased ability to do the tests. The authors propose that use of the same secret questions, under the same conditions every year, is the only way to maintain consistency.

  8.  The authors' main criticism is that the present system is open to the possibility of "drift", ie a change in standards that is not particularly significant from one year to the next, but which accumulates over time. There is quite a detailed discussion presented, which centres on the fact that the standard for the tests in a given year is based upon that for the previous year, rather than some absolute benchmark. In theory, one could expect some random fluctuation around a fixed level, but in practice there is a tendency for marks to be pushed upwards. Reasons for this include: the "benefit of the doubt" being given for borderline decisions, particularly in the light of the consequences of such decisions; the use of a "pre-test", taken by a smaller number of pupils, who are both younger and not under any pressure, and so likely to obtain lower marks; the return of scripts to schools, which lead to challenges to marking, all of which will be in connection with perceived harsh, not generous, marking.

  9.  Standards at A level are discussed, with particular reference to what amounted to a "quota" system for the different grades ABE until recent years, and the consequent problems caused by different standards of students opting for different subjects. A number of different studies were carried out, which are referred to, that lead to the conclusion that A level grades in different subjects are not equivalent. It is certainly true that more students are now achieving higher grades, but the authors do not draw a firm conclusion as to whether this is a result of declining standards or improved performance. They do, however, incline towards the former, basing their view largely on their own research.

  10.  As far as OFSTED is concerned, it is worth remembering that it is for the QCA to set and maintain standards for examinations and assessments at all levels. If there is a problem with falling standards then it is difficult to see, in the first instance, that OSFTED has contributed to this.

  11.  It is worth noting, with regard to QCA, that the authors specifically reject the cynical view that QCA are "quietly adjusting the cut-off marks in order to make sure that government targets are met". They believe QCA is attempting to "do an honest job" but they do not agree that their methods guarantee reliable maintenance of standards.

 


 
previous page contents

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2002
Prepared 2 July 2002