Select Committee on Health Appendices to the Minutes of Evidence


APPENDIX 14

Memorandum by The NHS Centre for Reviews and Dissemination (NC 65)

1.  SUMMARY

  1.1  NICE has worked hard to provide an infra-structure for the production of guidance for the NHS. A small organisation with a huge remit, it is to be applauded for aiming to be consultative and transparent in its workings, while maintaining academic rigour and delivering guidance.

  1.2  As a credible and independent organisation, NICE guidance needs to be based on high quality evidence from research and to be seen as unbiased. However, NICE guidance may lack credibility on several counts. Time constraints may adversely affect the quality of the technology assessment reports (TARs) that inform NICE guidance. There are also concerns about the transparency of the Technology Appraisal Committee's decision-making process. Moreover, despite its best efforts, NICE is hampered by the pharmaceutical industries' concerns about the confidentiality of commercial documents.

  1.3  NICE aims to provide a single national focus for appraisal and guidance. It also has a remit to co-ordinate and disseminate high quality evidence based information within the NHS. Under pressure to develop quickly an infra structure to support guidance development, NICE has yet to fully develop its dissemination strategy. Its dissemination strategy is currently under review. NICE needs to collaborate with other key stakeholders in co-ordinating the dissemination and distribution of good clinical and cost effectiveness information to the NHS and its consumers.

2.  INTRODUCTION

  2.1  The NHS Centre for Reviews and Dissemination (CRD) was set up in January 1994, and is core funded by the DH, through its R & D Division, to identify and review the results of good quality research and to disseminate the findings to decision makers (policy makers, health professionals and patients) in the NHS. By producing and disseminating high quality systematic reviews of health care interventions, CRD promotes clinical effectiveness and the use of cost effectiveness information to inform best practice.

  2.2  CRD is one of the six review teams producing technology assessment reports (TARs) for NICE. These reports contain clinical and cost effectiveness data to inform the NICE technology appraisal process, which results in NICE guidance on new and existing health technologies.

  2.3  NICE also funds CRD to produce Effective Health Care bulletins (EHCBs) and Effectiveness Matters (EMs). These publications summarise the findings of systematic reviews and are widely distributed in the NHS, to inform clinical decision-making. The publications form part of the NICE dissemination strategy.

3.  DOES NICE PROVIDE CLEAR AND CREDIBLE GUIDANCE?

  3.1  NICE has worked hard to provide an infra-structure for the production of guidance for the NHS. A small organisation with a huge remit, it has set up six National Collaborating Centres, to assist in the development of clinical guidelines and audit. There are also two new advisory units (the National Guidelines Support and Research Unit and the National Guidelines and Audit Patient Involvement Unit) and a Guidelines Advisory Committee. These bodies will advise on and support the development of clinical guidelines and future national audits.

  3.2  NICE has also developed and reviewed its technology appraisal process and has produced guidance documents to assist those who contribute to the process, and thus to the development of NICE guidance. In developing systems to underpin the guidance process NICE is to be applauded for committing itself to using rigorous methodology to develop evidence based guidance in processes that involve health professionals, patients and carers. The Institute aims to be consultative and transparent in its workings, while maintaining academic rigour and delivering guidance.

  3.3  As a credible and independent organisation, NICE guidance needs to be based on high quality evidence from research and to be seen as unbiased. However, NICE guidance may lack credibility on several counts. Time constraints may adversely effect the quality of technology assessment reports (TARs) that inform NICE guidance at a national level. There are also concerns about the transparency of the Technology Appraisal Committee's decision-making process. Moreover, despite its best efforts, NICE is hampered by the pharmaceutical industries' concerns about the confidentiality of commercial documents. These concerns are outlined in more detail below.

  3.4  Time constraints. Utilising rigorous methodology (1) CRD estimates that a high quality systematic review takes an average of 12-18 months to complete. NICE aims for a six month turnaround from commission to submission. Such timeframes cause huge problems for CRD staff throughout the review process.

  3.5  One of the key elements of good quality systematic review is its searches, where the aim is to identify as much relevant research as possible, to minimise the many forms of publication bias. The search for studies and data gathering is inevitably narrowed by time constraints, which means that important research information may be missed.

  3.6  The sheer volume of work within tight deadlines places high demands on research staff who are committed to maintaining high academic standards. Staff are concerned that quality could be jeopardised in meeting such stringent deadlines.

  3.7  The six month rapid review process includes assessment of industry submissions. While NICE has prepared guidance for manufacturers and sponsors in preparing a submission, drug companies typically supply vast quantities of data, from which it is often difficult to extract relevant data. There are concerns that staff may miss important information, which could affect research findings and thus NICE guidance.

  3.8  Peer review, an integral component of external quality assurance, is also affected by time constraints. Both protocols and the completed TAR are peer reviewed. The quick turnaround time means that both the research question and its findings may not be subjected to the detailed scrutiny that is desirable.

  3.9  Recommendation: NICE guidance would be more credible, if founded on a more extensive review of the evidence base. Such reviews demands longer timescales for completion of the work.

  3.10  The Appraisal Committee's decision making process. CRD recognises and welcomes the fact that NICE aims to involve many stakeholders in the development of guidance. CRD also recognises that NICE aims to be a transparent organisation. Nonetheless, it is not clear how the submissions of various stakeholders are weighted. On several occasions, CRD has noticed a disparity between the research findings submitted to the Appraisal Committee and that body's subsequent guidance.

  3.11  For example, orlistat, an anti-obesity drug, was approved by NICE for use in certain types of obese patients. Although most of the studies reviewed showed results which were statistically significant in favour of orlistat, average results indicated that an orlistat-treated patient weighing around 100kg at baseline would be likely to lose around 3kg more in body weight compared with a patient receiving placebo after one year of treatment. In clinical terms, such a difference may be deemed to be small, and possibly of limited clinical value. Although the issues of the clinical relevance of the findings was mentioned in the NICE guidance, it was not given much emphasis.

  3.12  A further example is the NICE recommendation that topotecan should be considered as one of the treatment options for women with advanced ovarian cancer, if first line chemotherapy has not been successful. The research evidence consists of only two randomised trials (and more studies of weaker designs) which compared topotecan with paclitaxel and caelyx respectively. All three drugs have considerable side effects, and the clinical significance of the findings has not been discussed in these studies. The systematic review concluded that there is no convincing evidence for the effectiveness of topotecan in the second line treatment of ovarian cancer, and that further studies are indicated.

  3.13  The research findings appear at times to receive less weight than submissions from other stakeholders. The lack of clarity and transparency in the decision making process may lend itself to accusations of bias in favour of certain aspects of the process (e.g, input by the pharmaceutical industry). The independence of NICE is threatened by this lack of clarity. NICE may be viewed as open to persuasion by various pressure groups—doctors, patients, government and the drug companies, all of whom have special interests to advocate or protect.

  3.14  Recommendation: The decision making process that concludes the work of the appraisal committee needs to be more transparent, with due weight given to the scientific review of research evidence.

  3.15  Confidentiality of consultation documents. NICE is aware that the concerns of the pharmaceutical industries about maintaining the confidentiality of consultation documents is matched by equal concern in the academic community about submissions of partial evidence from the pharmaceutical industries. Unlike submissions to the Medicines Control Agency, the pharmaceutical industry is not bound by law to disclose all information about a health technology in its submissions to NICE. This may render some of the evidence underpinning NICE national guidance incomplete. It may also threaten NICE guidance in that health professionals and the public may doubt the credibility of NICE recommendations if they are partly based on "commercial in confidence" data which is not in the public domain. It also threatens the independence of NICE, which may be seen as less powerful than the pharmaceutical industry, or even open to accusations of bias in favour of the industry. Moreover, it might be considered unethical to base national guidance on partial disclosure of information from an industry keen to protect and maintain its profits.

  3.16  Recommendation: The pharmaceutical industry should set up a voluntary agreement that all evidence will be submitted as part of the technology appraisal process, or the Government should introduce legislation compelling full disclosure of all evidence.

4.  HAS NICE ENDED CONFUSION BY PROVIDING A SINGLE NATIONAL FOCUS FOR APPRAISAL AND GUIDANCE?

  4.1  NICE was set up to provide a single national focus for appraisal and guidance and, with a high media profile and government backing, there is evidence to suggest that the NHS is setting up systems, through its clinical governance function, to take note of NICE guidance.

  4.2  In setting up six guidelines collaborating centres, two advisory units and a Guidelines Advisory Committee, NICE aims to manage the guidelines process and to maintain standards, while increasing the sense of ownership of guidance and devolving some of the work necessary to produce it. Through effective partnerships, guidance should be seen as the responsibility of a range of professional organisations, not just NICE.

  4.3  While NICE aims to provide a single national focus for appraisal and guidance, decision-makers in the NHS need to sift and weigh evidence, from a range of sources, to provide best practice. Other independent and well respected sources, such as the Drugs and Therapeutic Bulletin, the Effective Health Care publications, the Cochrane Library, and academic journals contribute to informed debate about effective health care. The National electronic Library for Health (NeLH) is developing as a gateway to evidence-based resources. NICE needs to develop further as an independent organisation working collaboratively with the many others committed to providing information to enable high standards of care.

  4.4  Recommendation: Set up a working group to facilitate effective partnership between information providers to the NHS. NICE and other stakeholders need to collaborate on horizon scanning, topic selection and coverage, targeted dissemination and the distribution of information to the NHS and its consumers.

5.  IS NICE PROVIDING GUIDANCE THAT IS LOCALLY OWNED AND ACTED ON IN THE RIGHT WAY?

  5.1  Effective implementation of guidance depends upon local ownership and available resources. The mandatory status of NICE guidance, with funding to follow, will go some way to making NICE guidance "everyone's business". However, NHS practitioners need to be persuaded of the validity of the guidance underpinning recommendations for best practice. The fact that NICE guidance is based on systematic reviews and NICE consults interested parties from commissioning through to publication will help ensure wide ownership of guidance. However, the decision making process which concludes the work of the appraisal committee, needs to be more transparent, to ensure credibility and thus take up. (See Recommendation, paragraph 3.14, above).

6.  IS NICE ACTIVELY PROMOTING INTERVENTIONS WITH GOOD EVIDENCE OF CLINICAL AND COST EFFECTIVENESS SO THAT PATIENTS HAVE FASTER ACCESS TO TREATMENTS KNOWN TO WORK?

  6.1  NICE has a remit to co-ordinate and disseminate high quality evidence based information within the NHS. In doing this it actively promotes interventions with good evidence of clinical and cost effectiveness. NICE supports National Cancer Service Guidance, Prodigy, MeReC publications and Effective Health Care publications as part of its dissemination strategy (3).

  6.2  CRD's Effective Health Care bulletins (EHCBs) are succinct summaries of research findings from up to date systematic reviews. They tend to focus on the clinical effectiveness, cost effectiveness and acceptability of health service interventions, such as the effectiveness of laxatives in adults (enclosure). The bulletins also focus on the evidence base underpinning guidance such as the National Cancer Service Guidance (enclosure). The bulletins are disseminated to a target audience of some 60,000 health service decision-makers. They are well regarded, with high levels of impact. A recent paper in the BMJ indicated that at a cost of £25,000, the bulletin on glue ear may have saved the NHS some £25 million (2).

  6.3  Despite good relations between the two organisations, the re-positioning of the bulletins within the NICE range of publications has proved problematic. Due to problems with the NICE contracting and commissioning process, the planning and production of the bulletins programme has been difficult, so that CRD has struggled to meet its contractual commitment to publish six EHCBs and two Effectiveness Matters per financial year. The lack of a regular publication schedule has impacted on CRD's dissemination activity and on some Trusts' clinical audit and effectiveness agendas.

  6.4  Under pressure to develop quickly an infra structure to support guidance development, NICE has yet to fully develop its dissemination strategy, which has, in turn, affected CRD's dissemination activity. The NICE dissemination strategy is now under review, suggesting that the organisation is aware of the need for its further development. As the national Centre for Reviews and Dissemination, CRD staff have several years experience of targeting information to policy makers, health professionals and the public. CRD is, of course, willing to collaborate with NICE in promoting clinical and cost effectiveness information to the NHS and its consumers.

  6.5  Recommendation: NICE and CRD to formally review their contract and commissioning process.

  6.6  Recommendation: NICE needs to collaborate with CRD and other key stakeholders in co-ordinating the dissemination and distribution of evidence based information to the NHS and its consumers. (see also Recommendation paragraph 4.4 above)

  6.7  REFERENCES

  1.  NHS Centre for Reviews and Dissemination. Undertaking Systematic Reviews of Research on Effectiveness: CRD's Guidance for Carrying Out or Commissioning Reviews. (CRD Report Number 4) (2nd Edition). York: NHS Centre for Reviews and Dissemination, 2001.

  2.  Mason J, Freemantle N and Browning G. Impact of Effective Health Care bulletin on treatment of persistent glue ear in children: time series analysis. British Medical Journal 2001; 323:1096—97.

  3.  National Institute for Clinical Excellence. Annual Report 2000-01 and Summary Financial Statement. National Institute for Clinical Excellence, September 2001.



 
previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2002
Prepared 8 July 2002