Select Committee on Health Minutes of Evidence

Memorandum by Sir Iain Chalmers (PI 29)


  I am editor of The James Lind Library, a public website explaining and documenting the evolution of fair tests of treatments in health care ( I am also a co-convenor of the James Lind Alliance, a coalition of organisations representing patients and clinicians collaborating to confront important, shared uncertainties about the effects of treatments in health care ( I have a longstanding interest in improving the quality of evidence about the effects of treatments in health care, an interest that was prompted initially by realising that I had been harming my patients by relying on "eminence-based" rather than "evidence-based" guidance. I have previously submitted evidence to the committee for its enquiry into the National Institute for Clinical Excellence (NICE). I am submitting this memorandum in an individual capacity. I would be willing to give oral evidence, if the Committee decides that this might be helpful.


  Many patients are suffering and dying unnecessarily, and resources for health care and research are being wasted, because of:

    (1)  perverse influences on the clinical research agenda;

    (2)  failure to take systematic account of existing knowledge when planning clinical research;

    (3)  biased under-reporting of research;

    (4)  insufficient public and professional access to trustworthy evidence about treatment effects and information about relevant ongoing research;

    (5)  government-endorsed secrecy about the effects of licensed drugs; and

    (6)  the introduction of inadequately assessed treatments within the NHS.

   The pharmaceutical industry inevitably contributes substantially to these serious problems because of the very prominent role it plays in clinical research and information about the effects of treatments.

  This unsatisfactory state of affairs can be addressed by:

    (1)  assessing the extent to which the clinical research agenda reflects the priorities of patients and practitioners for developing and evaluating healthcare innovations;

    (2)  ensuring that systematic reviews of existing evidence are the basis for planning new research;

    (3)  requiring public registration of clinical trials with unique identifiers and outlawing biased under-reporting of health research;

    (4)  increasing the capacity for preparing, maintaining and disseminating systematic reviews of the effects of treatments, and for providing public information about ongoing research;

    (5)  promoting public access to the results of all clinical studies of the effects of licensed drugs; and

    (6)  restricting the use of inadequately evaluated drugs within the NHS to the context of controlled evaluative studies, thus reducing unacceptable uncertainties about their cost-effectiveness.


  Over the past half century, better health care has been responsible for between a third and a half of the increase in life expectancy, and an average of five additional years free of chronic health problems (Bunker et al 1994). Compared with all previous eras of health care, this is a remarkable record of success, and drug treatments have made their contribution to it. Indeed, the extent of these achievements over the 50 years before the human genome was sequenced will be the yardstick against which we will come to judge the validity of current claims that the genetic revolution promises even greater benefits to patients.

  In spite of these heartening estimates of the effects of health care, the public could have obtained better value for the substantial private and public resources invested in research intended to improve health, particularly during the past 20 years. During this time, very few newly approved drugs have offered worthwhile advantages to patients over previously available options (International Society of Drug Bulletins 2001), a problem that has been noted within the pharmaceutical industry (for example, Horrobin 2003) as well as outside it (for example, Garattini and Bertele 2002).

  Over the past two decades, industry and academia have increasingly worked in partnership to promote commercially dictated objectives. This has had perverse influences on the clinical research agenda (Chalmers 2000). For example, most clinical trials relating to osteoarthritis of the knee are commercial studies of drugs. When patients, rheumatologists, physiotherapists and general practitioners were asked to identify their priorities for research, they made clear that they had little need for any more such studies, but instead wanted more rigorous evaluations of physiotherapy and surgery, and assessments of the educational and coping strategies that might help patients to manage this chronic, disabling and often painful condition more successfully (Tallon et al 2000).

  Studies evaluating the effects of out-of-patent drugs and non-drug interventions cannot expect to attract support from industry. The opportunity costs of acquiescing in industry's pervasive influence on the clinical research agenda is that many questions of importance to patients, professionals and the NHS more generally, are not being addressed.


  Independent evaluations should be commissioned to assess the extent to which research reflecting the priorities of the pharmaceutical industry is promoting or jeopardising the development and evaluation of innovations regarded as important by patients, practitioners and the NHS itself.


  An editorial in the British Medical Journal entitled "the scandal of poor medical research" has called for "less research, better research and research done for the right reasons" (Altman 1994). In part, the scandal reflects the perverse influences of commercial interests on the research agenda. For example, there is evidence suggesting that some studies sponsored by industry have been designed deliberately to yield results favourable to particular drugs (Djulbegovic et al 2000). This can be done by withholding a comparison treatment known to help patients, or giving comparison treatments in inappropriately low or high doses.

  The scandal of poor medical research would be reduced if researchers were required to take systematic account of existing knowledge when planning and reporting their research. Because this job is not done well, patients have been denied treatments which could have benefited them, while other patients have received treatments that were either of no material benefit, or actually harmful (Antman et al 1992).

  For example, after reviewing the experience of thousands of patients who had participated in controlled trials of new calcium-blocking drugs given to people experiencing a stroke, a Dutch team found no evidence to support the increasing use of these drugs in practice, or for the large numbers of clinical trials that had been performed (Horn and Limburg 2001). Furthermore, when they subsequently prepared a systematic review of the relevant animal studies they found that these had never suggested that the drug would be useful in humans (Horn et al 2001).

  These serious problems would be reduced if all funders of research followed the lead of the Medical Research Council in requiring those seeking support for new research to refer explicitly to systematic reviews of existing evidence, showing that proposed additional research is necessary and that it is building on previous experience. Because research ethics committees have tended to ignore this principle, patients and the public have not been protected effectively from unethical research (Savulescu et al 1996).


  As a precondition for funding and ethical approval, funders and research ethics committees should be required to ensure that proposals for new research are supported by systematic reviews of existing evidence.


  Reliable evidence about the effects of drugs on patients depends on systematic reviews of all relevant, unbiased studies. Such systematic reviews are now used extensively by the National Institute for Clinical Excellence (NICE) and in clinical guidelines. The increased amount of evidence being assembled in systematic reviews is a major advance. However, these reviews may overestimate the beneficial effects of drugs and underestimate their adverse effects because the results of some key studies are not publicly available: studies that have not found beneficial effects of drugs, or which have found adverse effects, are less likely than others to be published (Dickersin 1997).

  Biased under-reporting of clinical trials kills patients and wastes resources. For example, had all the completed studies of drugs to reduce heart rhythm abnormalities in patients having heart attacks been reported, tens of thousands of deaths from these drugs could have been avoided. To their credit, Dr Cowley and his colleagues in Nottingham pointed out how an unpublished study done there 13 years previously might have provided an early warning of this iatrogenic disaster. "When we carried out our study in 1980", they reported, "we thought that the increased death rate was an effect of chance—The development of lorcainide was abandoned for commercial reasons, and this study was therefore never published; it is now a good example of "publication bias" (Cowley et al 1993).


  All clinical trials done within the NHS should be registered publicly at inception, and assigned an International Standard Randomised Controlled Trial Number (

  Pharmaceutical companies wishing to perform trials in the UK should be required to publicly endorse and comply with the Good Publication Practice for Pharmaceutical Companies (Wager et al 2003 and

  Consideration should be given to introducing legislation—comparable to that in Spain—requiring pharmaceutical companies to publish all results of all randomised clinical trials involving NHS patients.


  Health professionals and patients alike face an overload of information about the effects of treatments. This problem is compounded by the fact that it is usually impossible to interpret the significance of the findings of individual research studies because researchers rarely set the results of new research in the context of systematic reviews of all other relevant data (Clarke et al 2002).

  Welcome support for the training and other infrastructure needed to prepare and maintain systematic reviews has been provided by all UK governments since 1991, largely through the NHS R&D Programme and NICE. The results of these analyses have also been increasingly successfully made available for professionals and patients, for example through the Cochrane Library, The Centre for Reviews and Dissemination, the NHS Health Technology Assessment Programme, NICE, NHS Direct, and the National electronic Library for Health.

  In spite of this encouraging progress, however, current levels of support for preparing and maintaining systematic reviews remains dangerously inadequate. Professionals and patients have ready access to only a small proportion of existing evidence about the effects of drugs and other treatments. As systematic reviews of existing evidence are an essential (albeit not sufficient) component of the foundation for informing professional practice, patient choice, and policies within the NHS, support for this work must be increased substantially.

  Furthermore, when systematic reviews of existing evidence reveal that there are important uncertainties about the merits of treatments, healthcare professionals and patients should be able to find out more easily whether any ongoing research is addressing these uncertainties, and how they may be able to contribute to these studies.


  The Department of Health, the Medical Research Council and the medical research charities should increase substantially their current levels of investment in systematic reviews.

  The Department of Health should ensure that patient-friendly information about ongoing clinical trials within the NHS is made readily available.


  Continued lack of public access to information about the effects of licensed drugs is incompatible both with current public expectations and with the government's professed commitment to promoting informed patient choice and rational use of resources within the NHS. Denial of access to information held by the Medicines and Healthcare products Regulatory Agency (MHRA) puts the interests of pharmaceutical companies ahead of those of patients and prescribers. This is particularly indefensible in the light of evidence of that regulatory agencies, supposedly established to protect the public, are acquiescing in biased later publication of the information they hold (Melander et al 2004).

  For example, after the safety of human albumin solution had been called into question in a systematic review of published and unpublished controlled trials, the Medicines Control Agency (MCA, which preceded the MHRA) refused to disclose the evidence that had led it to grant a licence for the product (Roberts et al 1998). The government colluded with the manufacturers of evening primrose oil in suppressing information suggesting that the drug had no useful effect in eczema. Five years later, the MCA withdrew the marketing authorisation for evening primrose oil on grounds of lack of efficacy, but without revealing the evidence upon which it had based its decisions initially to award and then to withdraw a licence for the drug (Chalmers 2004).


  There should be easy public access to all information from clinical studies of the effects of licensed drugs, including that held by the Medicines and Healthcare products Regulatory Agency.


  During the 2nd World War, a drug was produced which was thought to promote faster recovery from common colds—a substantial cause of lost work and reduced national efficiency. Because of its potential national importance, the Ministry of Supply asked the Medical Research Council to evaluate the drug. The pharmaceutical industry provided supplies of the drug and placebos for this research, and, within a few months, strong evidence had been produced showing that the drug was highly unlikely to have any useful effect (Clarke 2004). This showed how concerted common purpose can reliably answer questions of national importance about the effect of a drug. It is relevant today, and is reflected in the creation of NICE.

  NICE exists because the information supplied by pharmaceutical manufacturers to the MHRA is inadequate for assessing the extent to which the costs of drugs represent good value to the NHS. As I noted in evidence submitted to the Committee for its enquiry into NICE, the organisation cannot be expected to serve the interests of the public effectively while it does not have full access to relevant evidence, and while there is continued acquiescence in biased and incomplete public access to the results of relevant clinical research.

  NICE cannot make adequate assessments of cost-effectiveness without information about the relative merits of alternative treatments (including drugs in the same class), and evidence gathered after wider use and longer follow up of new products showing their effects on the outcomes that matter to patients. In these circumstances of significant ignorance about the effects and cost-effectiveness of drugs, the NHS should consider adopting recommendations made in respect of the Australian Pharmaceutical Benefits Scheme (Glasziou 1995): promising but inadequately evaluated drugs should be used by the NHS only within the context of research designed to reduce uncertainties about their value, and thus provide the basis for informed choices and decisions.


  Selected promising but inadequately evaluated drugs should be used in the NHS only within the context of controlled evaluative studies until enough is known to judge their cost-effectiveness.

14 August 2004


  Altman DG (1994). The scandal of poor medical research. BMJ 308:283-4.

  Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC (1992). A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. JAMA 268:240-48.

  Bunker JP, Frazier HS, Mosteller F (1994). Improving health: measuring effects of medical care. Milbank Quarterly 72:225-258.

  Chalmers I (2000). Current Controlled Trials: an opportunity to help improve the quality of clinical research. Current Controlled Trials in Cardiovascular Medicine 1:3-8. Available:

  Chalmers I (2004). In the dark. Drug companies should be forced to publish all the results of clinical trials. How else can we know the truth about their products. New Scientist, 6 March:19.

  Chalmers I, Hedges LV, Cooper H (2002). A brief history of research synthesis. Evaluation and the Health Professions 25:12-37.

  Clarke M. The 1944 patulin trial of the British Medical Research Council: an example of how concerted common purpose can get reliable answers to important questions very quickly. The James Lind Library (

  Clarke M, Alderson P, Chalmers I (2002). Discussion sections in reports of controlled trials published in general medical journals. JAMA 287:2799-2801.

  Cowley AJ, Skene A, Stainer, Hampton JR (1993). The effect of lorcainide on arrhythmias and survival in patients with acute myocardial infarction. International Journal of Cardiology 40:161-166.

  Djulbegovic B, Lacevic M, Cantor A, Fields KK, Bennett CL, Adams JR, Kuderer NM, Lyman GH (2000). The uncertainty principle and industry-sponsored research. Lancet 356:635-638.

  Garattini S, Bertele V (2002). Efficacy, safety, and cost of new anticancer drugs. BMJ 325:269-71.

  Glasziou PP (1995). Support for trials of promising medications through the Pharmaceutical Benefits Scheme. Medical Journal of Australia 162:33-36.

  Horn J, Limburg M (2001). Calcium antagonists for acute ischemic stroke (Cochrane Review). In: The Cochrane Library, Issue 3, Oxford: Update Software.

  Horn J, de Haan RJ, Vermeulen M, Luiten PGM, Limburg M (2001). Nimodipine in animal model experiments of focal cerebral ischaemia: a systematic review. Stroke 32:2433-38.

  Horrobin DF (2003). Modern biomedical research: an internally self-consistent universe with little contact with medical reality. Nature Reviews/ Drug Discovery 2:151-154.

  International Society of Drug Bulletins (2001). ISDB Declaration on therapeutic advance in the use of medicines.

  Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B (2003). Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ 326:1171-3.

  Roberts I, Li Wan Po A, Chalmers I (1998). Intellectual property, drug licensing, freedom of information, and public health. Lancet 352:726-729.

  Savulescu J, Chalmers I, Blunt J (1996). Are research ethics committees behaving unethically? Some suggestions for improving performance and accountability. BMJ 313:1390-1393.

  Tallon D, Chard J, Dieppe P. Relation between agendas of the research community and the research consumer. Lancet 2000;355:2037-40.

  Wager E, Field EA, Grossman L (2003). Good Publication Practice for pharmaceutical companies. Current Medical Research and Opinion 19:149-54, and at

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2005
Prepared 26 April 2005