Select Committee on Public Administration Memoranda

Memorandum by the British Medical Association (PST 1)

Target-setting and performance measurement


1.  Target-setting can be regarded as shorthand for the wide range of performance measures, guidelines and service frameworks and agreements, which underpin the delivery of public services especially health services. The current NHS performance ratings and high level performance indicators set within the broader performance assessment framework are the latest in a long line of performance measures which began in the 1980s with activity and cost indicators and progressed through efficiency indicators and patient charters in the 1990s.

The present system

2.  At present, most of the debate centres on the performance ratings and their role in determining access to 'earned autonomy.' NHS providers have their performance assessed against a limited number of key targets and a larger number and range of indicators. For acute trusts the key targets are as follows:

  • no patients waiting more than 18 months for inpatient treatment
  • fewer patients waiting more than 15 months for inpatient treatment
  • no patients waiting more than 26 weeks for outpatient treatment
  • fewer patients waiting on trolleys for more than 12 hours
  • less than 1% of operations cancelled on the day
  • no patients with suspected cancer waiting more than two weeks to be seen in hospital
  • improvement to the working lives of staff
  • hospital cleanliness
  • a satisfactory financial position

3.  The broader range of indicators is intended to provide a balanced view across three focus areas - clinical, patient and capacity/capability. Examples of clinical focus indicators are risk of clinical negligence, post-operative death, speed of discharge and emergency re-admission to hospital. Examples of patient focus indicators are inpatient, outpatient and A & E waits, delayed discharges and patient satisfaction. Examples of capacity/capability focus include data quality, staff satisfaction compliance with the New Deal on junior doctors' hours and the sickness/absence rate for directly employed NHS staff. A Trust, which has demonstrated high standards of performance against the key targets and in these three areas, will receive a performance rating of three stars.

The purpose of target-setting

4.  The government's aims in constructing its performance ratings can be summarised as follows:

  • to monitor progress in what it has identified as key policy areas
  • to set targets in these areas and
  • to incentivise and support progress towards these targets.

5.  The incentives it uses or intends to use are the right to autonomy and more direct financial incentives through changes to staff contracts.

6.  In a discussion document published in March 2000,[1] the BMA criticised earlier performance indicators largely because their audience was ambiguous. If the audience was the patient then they were overly complex and would not satisfy the public's desire for simple easy to understand relative information at a disaggregated level. If their audience was to be the individual clinician then they were insufficiently specified for local circumstances including case mix and relative risk. They were probably nearer to satisfying the necessary criteria for use by commissioning bodies and/or NHS Trusts to assess relative performance and to identify areas for further exploration. This could, in due course, result in dissemination of best practice and as a result lead to improvements in the overall quality of health care delivery. However, crucially, the measures were probably too insensitive to variations in inputs. Differences in staff mix and in other resources were not sufficiently controlled for. These reservations continue to apply to the present measures.

7.  One dilemma faced by successive governments has been to reconcile local and clinical autonomy with central control. The present resource allocation arrangements go further than their predecessors, which were designed to provide equal resources and thus equal access to health care for those in equal need. They now explicitly include resources aimed at health inequalities in addition. This argues for tailoring services to specific local circumstances where these currently contribute to health inequalities. However, the government is reluctant to simply set unified budgets (including this element) for commissioners and leave them to decide how best to use these to pursue a broad agenda. Earmarked funding and performance measures are ways of retaining central control.

8.  This does not necessarily run counter to patient wishes. In research underpinning the BMA's Healthcare Funding Review, we found strong public support for the basic concept of a healthcare system which is essentially free at the point of use and aims to provide equal access to the same standard of care for all. This suggests that the public would react adversely to local provision that created different expectations in different areas. Some measure of uniformity is thus necessary.

What sort of approach to performance measurement could we support?

9.  Performance measurement is a desirable process provided certain basic underlying principles are met:

  • There should be support and consensus for the key policy areas concerned
  • The targets should be relevant to these policies
  • There should be a small number of meaningful targets
  • The targets should be within the capacity of receivers to address
  • Any behavioural change stimulated by the targets should be desirable
  • The measures used to assess progress should address patient and clinician concerns
  • The measures used should focus on outcome rather than process
  • The process itself should be one of self comparison and benchmarking rather than ranking

10.  The present arrangements fall short of these principles. By and large there is support for policies aimed at ending unacceptable variations in health outcomes, but the present targets are more about process than outcome. Where outputs and processes are proxies for outcomes, they should be meaningful ones. The problem is that true outcome measures need to be risk adjusted and to control for case mix and input variation.

11.  The existing measures are, arguably, overly complex and resource intensive. To influence behaviour, targets must be personally involving and relevant, such that the receivers regard the recommendation as applicable to their situation and needs. Many of the output measures, particularly those in the public service agreements are beyond the capacity of those in the service to deliver. For example, of those set out in the health department public service agreement (PSA), only efficiency savings, waiting list and prescribing targets together possibly with the admission rates for elderly and psychiatric patients fall remotely into this category. Some require inter-agency co-operation (e.g. those dealing with delayed discharge) or are better addressed with through non-health services (e.g. the prevention of certain life-threatening conditions).

12.  There is a danger in performance management systems for the process to become more important than the outcome. Receivers may well manage their workloads to satisfy the narrowly defined targets, with perverse results. Maximum waiting times for specific diagnoses may prompt over-referral for example.

13.  The government has indicated that in general the indicators should take into account data availability and make use of existing data where possible. There may, however, be better indicators and the balance between quality and cost should be borne in mind. The set of indicators should be as small as possible rather than attempting to cover too much ground. Again, it is a question of balance - this time between manageability and coverage.

14.  As our discussion document (see above) pointed out, experience of league tables and their use in the education sector suggests that consumers and professionals in the service see them very differently. To the former, relative position is paramount whereas to the latter performance is best measured by added value. Good teaching can increase the performance of individual pupils in a school with little effect on overall ranking and thus on consumer reaction. This has led to the development of alternative performance indicators for schools produced by non-government agencies. These feed back to local education authorities a wide variety of measures based on added value. These measures share a number of characteristics. They are multilevel analyses on a range of indicators and all control for background factors including most importantly prior attainment. The analogy with health services is clear. NHS staff need feedback about the effectiveness of treatments and processes and patients need information to manage expectations. However, patients are in different states of health when they access the service and outcomes depend heavily on this.

15.  When we produce targets and measure movement towards them, we should bear in mind the ultimate user of this information - the patient. Patients tell us that they require close proximity to high quality health services. Performance measurement should therefore be aimed at identifying and promoting best practice using benchmarking and feedback. Choice depends for its existence on variation Emphasising performance ratings and patient choice may well therefore be counter-productive.

October 2002

1   Clinical indicators (league tables): a discussion document. BMA Board of Science and Education. March 2000


previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2002
Prepared 7 November 2002