Science and Technology CommitteeWritten evidence submitted by Sir Iain Chalmers
1. How could the occurrence of clinical trials be made more open to scrutiny?
1.1 Government regulation should be introduced requiring all clinical trials, together with their protocols, to be registered publicly at inception (Chalmers 2004a).
1.2 Patient-friendly information should be available for all trials open to recruitment, as it is already for all cancer trials (Godlee and Chalmers 2010).
2. How could the results of clinical trials be made more open to scrutiny?
2.1 People being invited to participate in controlled trials should require written assurance that the full study results will be published, and that these will be sent to all participants who indicate that they wish to receive them (Evans et al, 2011; www.testingtreatments.org).
2.2 Regulation is needed to ensure that all clinical trials are published (Chalmers 2004a; www.alltrials.net), and that information identifying sponsors, institutions and individuals who have failed to publish registered trials is also published, with either acceptable explanations or resultant sanctions.
2.3 All clinical trials should be published, regardless of the type of intervention(s) evaluated, and whether they are commercially sponsored or non-commercially sponsored (Chalmers et al, 2012). A focus on regulation of the pharmaceutical industry cannot be expected to have any impact on non-publication of trials of interventions other than medicines.
2.4 The academic journal system cannot be relied upon to deal with the problem of under-reporting of research (Smith 2006). Trial registration provides the most appropriate alternative framework for publishing the results of clinical trials.
3. Personal background: Three decades of failure to promote real change
3.1 I am a clinically qualified health services researcher, currently responsible for coordinating the work of the James Lind Initiative (JLI). The JLI has been funded by the National Institute for Health Research to promote acknowledgement of uncertainties about the effects of treatments and research to address them.
3.2 Biased under-reporting of research results in avoidable suffering and deaths of patients and waste of resources in health care and health research (Chalmers and Glasziou 2009). I have been concerned about the scientific and ethical consequences of biased under-reporting of research since the early 1980s (Grant and Chalmers 1981). In a letter published in the BMJ in 1985 I proposed that the term “negative trial” should be outlawed, because “All trials that have been well conceived and well conducted—whatever their results—represent positive contributions to knowledge” (Chalmers 1985).
3.3 Since the early 1990s, I have emphasised that “failure to provide adequate, publically available reports of the results of clinical trials does an injustice to the patients who have participated in them, as well as to others who have collaborated with the investigators and those who have provided funds or other resources” (Chalmers 1990). From the mid-1990s onwards I have challenged research ethics committees to use their regulatory influence to reduce this problem (Savulescu et al, 1996; Pearn and Chalmers 1996; Chalmers 1997; Roberts et al, 1998; Chalmers 2002; Antes and Chalmers 2003; Smith and Chalmers 2007; Garattini and Chalmers 2009). I have also challenged professional organisations—the Academy of Medical Sciences and the Royal College of Physicians of London in particular—to follow the lead of the Faculty of Pharmaceutical Medicine in deeming it unethical to acquiesce in under-reporting of research. There is little evidence that the issue has been taken seriously by research ethics committees or professional organisations.
3.4 The inquiry by the House of Commons Health Committee into the Influence of the Pharmaceutical Industry in 2004 provided an opportunity to draw the problem to the attention of parliamentarians and I submitted written evidence and gave oral evidence to the Committee (Chalmers 2005). I have subsequently raised the problem of biased under-reporting of research with parliamentarians through an article in Science in Parliament (Chalmers 2007) and evidence submitted to the Health Committee’s inquiry into “Aspects of the work of the National Institute of Health and Clinical Excellence” (Evans et al, 2007), and through the Science and Technology Committee’s inquiry into “Peer review in scientific publications” (Chalmers 2011).
3.5 My attempts over 30 years to persuade researchers, research funders, professional organisations, research ethics committees, parliamentarians and governments to take this issue seriously have not been successful, however. A few years ago I wrote an article entitled “From optimism to disillusion about commitment to transparency in the medico-industrial complex” (Chalmers 2006a). In it, I drew attention to efforts made by some individuals and organisations during the 1990s to address the problem of biased under-reporting, but I also referred to the emergence of increasing evidence that fundamental problems remained and that the situation might actually be getting worse. I ended the article by expressing my hope that I might be able to write another essay in five years entitled “From disillusion to optimism in about the scientific integrity of the pharmaceutical industry and the people collaborating with it”.
3.6 My approach since then has been to try to increase public awareness of how the public is being “sold short” just as long as half the studies to which they have contributed are not being reported (Chalmers 2004a; 2006b). In 2006, colleagues and I published a book for the public to increase general knowledge about why it is important to test treatments rigorously, and how to recognise inadequate evidence, including incomplete evidence (Evans et al, 2006). The book was translated into six other languages, a second edition was published in 2011 (Evans et al, 2011), and it is now the foundation of a website called Testing Treatments interactive which makes available video and audio material and other resources helping to illustrate the concepts covered in the book (www.testingtreatments.org). Both editions of the book (and the website) have a suggested Action Plan for its readers. Among other things, this suggests that they should:
Encourage and work with health professionals, researchers, research funders, and others who are trying to promote research addressing inadequately answered questions about the effects of treatment which you regard as important.
Agree to participate in a clinical trial only on condition (i) that the study protocol has been registered and made publicly available (ii) that the protocol refers to systematic reviews of existing evidence showing that the trial is justified; and (iii) that you receive a written assurance that the full study results will be published, and sent to all participants who indicate that they wish to receive them.
3.7 I am hopeful that making the public more aware of the scandal of under-reporting of research will help to bring about the changes needed, despite the very powerful forces that will continue to defend the status quo.
3.8 Jeremy Paxman summed up the current situation in a word. On Wednesday 27 July 2011 there was a discussion on Newsnight about the Bateson review of research using non-human primates. Susan Watts’ introductory package noted that the review made clear that “those using primates should publish any negative results, to prevent work being repeated unnecessarily.” Paxman’s interviewees were Paul Matthews, a member of the Bateson Review Group, and Tipo Aziz, Professor of Neurosurgery at Oxford University.
Matthews: “There is one other point that is important to bear in mind. Negative results are not results of no value.”
Paxman: But they’re results of no value if noone knows about them.
Matthews: … This is what the committee felt very strongly needed to be part of the change that we help to drive forward from now on.
Paxman: So what you’re saying is, that if you don’t get the result you’re looking for, or a result you consider to be of any use, you should nonetheless publish it so that others know.
Matthews: Absolutely. If you ask a good question, a positive result is of value and a negative result is of value.
Paxman: Why doesn’t that happen already?
Aziz: For several reasons. If one achieves a negative result very few journals will publish it.
Paxman: Surely, on the web anyone can publish anything.
Aziz: Yes, but perhaps not in the most respected journals, one that would bring impact or cite your work. The other thing is, publishing negative work also detracts from your chances of getting further research funding.
Paxman: What? If you admit that it didn’t work out you might not get paid to do it again?
Aziz: Not the same experiment again, but to do further research along those lines.
Paxman: That’s nuts isn’t it?
3.9 I hope that the Science and Technology Committee will agree with Jeremy Paxman that the current situation is indeed “nuts”—unethical, unscientific and uneconomic nuts.
3.10 My efforts to prompt improvement in clinical trial transparency over most of the past 30 years have manifestly failed. However, it is becoming clear that Sense about Science’s recently launched public campaign (www.alltrials.net) and Ben Goldacre’s bestselling book Bad Pharma may be “game changers”. For the first time in over 30 years I feel that there is reason to hope for substantive progress. I think that those who continue not to take under-reporting of research seriously will find themselves on the wrong side of history. I hope that the Committee will see to it that, after decades of inadequate action, something substantial will be done to deal with the current, indefensible situation.
Antes G, Chalmers I (2003). Under-reporting of clinical trials is unethical. Lancet 2003;361:978–979.
Chalmers I (1985). Proposal to outlaw the term “negative trial”. BMJ 290:1002.
Chalmers I (1990). Under-reporting research is scientific misconduct. JAMA 1990;263:1405–1408.
Chalmers I (1997). Fraud and misconduct in medical research. Journal of the Royal College of Physicians of London 1997;31:216–217 (book revew).
Chalmers I (2002). All unbiased comparative studies should be published. BMJ 324:483.
Chalmers I (2004a). Government regulation is needed to prevent biased under-reporting of clinical trials. BMJ 329:462.
Chalmers I (2004b). In the dark. Drug companies should be forced to publish all the results of clinical trials. How else can we know the truth about their products. New Scientist 2004, 6 March:19.
Chalmers I (2005). Written and oral evidence. In: House of Commons Health Committee. The Influence of the Pharmaceutical Industry. Fourth Report of the Session 2004–05, Vol. II. London: Stationery Office, Ev 194–208.
Chalmers I (2006a). From optimism to disillusion about commitment to transparency in the medico-industrial complex. Journal of the Royal Society of Medicine 2006;99:337–341.
Chalmers I (2006b). Biomedical research: are we getting value for money? Significance 2006;3:172–75.
Chalmers I (2007). When does clinical science cease to exist? Science in Parliament 2007;64: 19–20.
Chalmers I (2011). Written evidence to Science and Technology Committee (PR47)—Eighth Report. Peer review in scientific publications. Ordered by the House of Commons to be printed 18 July 2011.
Chalmers I, Glasziou P (2009). Avoidable waste in the production and reporting of research evidence. Lancet 374:86–89.
Chalmers I, Glasziou P, Godlee (2012). All trials must be registered and the results published: academics and non-commercial funders are just as guilty as industry. BMJ 346:f105.
Evans I, Thornton H, Chalmers I (2006). Testing Treatments. London: British Library.
Evans I, Thornton H, Chalmers I (2007). Submission to the House of Commons Health Committee inquiry into aspects of the work of the National Institute of Health and Clinical Excellence (NICE).
Evans I, Thornton H, Chalmers I, Glasziou P (2011). Testing Treatments. London: Pinter and Martin. Downloadable for free at www.testingtreatments.org.
Garattini S, Chalmers I (2009). Patients and the public deserve big changes in evaluation of drugs. BMJ 338:804–806.
Godlee F, Chalmers I (2010). Information about ongoing clinical trials for patients. BMJ 2010;340:456–57.
Grant A, Chalmers I (1981). Register of controlled trials in perinatal medicine. Lancet 1981;1:100.
Pearn J, Chalmers I (1996). Is selective reporting of well-designed clinical research unethical as well as unscientific? Nederlands Tijdschrift voor Geneeskunde 140:220–221.
Roberts I, Li Wan Po A, Chalmers I (1998). Intellectual property, drug licensing, freedom of information, and public health. Lancet 1998;352:726–729.
Savulescu J, Chalmers I, Blunt J (1996). Are research ethics committees behaving unethically? Some suggestions for improving performance and accountability. BMJ 1996;313:1390–1393.
Smith R (2006). The trouble with medical journals. London: RSM Press.
Smith R, Chalmers I (2007). Ethics and medical publishing. In: Ashcroft R, Dawson A, Draper H, McMillan J, eds. Principles of Health Care Ethics. Chichester: John Wiley, 2007: 751–758.