Examination of Witnesses (Questions 1
WEDNESDAY 15 NOVEMBER 2000
1. Welcome to the Committee for the return bout,
and thank you for your memorandum. We are flattered that you quoted
so extensively from our report in your article in the Veterinary
Record of February 19a word-for-word reference to our
reportand regret only that we were not acknowledged. I
suppose we will simply be content with the implied approval. Let
us start with the obvious question, if we may. How confident are
you that the programme you are overseeing is going to deliver
results, and when will we know what they are?
(Professor Bourne) I think at the last
Select Committee meeting you did not have our second report available
to you, but you subsequently had that and, of course, you would
have read that and noted that the timetable that we recommended
should be put in place and wished to adhere to has been met by
MAFF with respect to the proactive culls. We recommended that
seven should be completed by the end of this trapping year, which
terminates at the end of January, with the final three being completed
next year, and all 10 triplets have now been identified. We are
up to the time-scale on that. We went into some detail in the
second report on the likely timing of the emergence of results.
You will gather from the report that we do not envisage a time
when one can open a window and see everything clearly. What we
will see is the accumulation of analyses of data which starts
now, from which will emerge knowledge on which policy options
will certainly, be based in future. However, the impact that knowledge
will have on policy options in the short term, of course, is not
known. For instance, we have started looking at TB99 data. Our
intention is that an annex will accompany the next report we write,
which will be in March, and from that we will gain a little more
information with respect to some of the risk factors. There will
be nothing earth-shattering about that, but it will be the start
of knowledge being gained and made available to the general public
and, of course, to Government, which may or may not influence
policy. With respect to the trial itself, we again went into some
detail and, as you know, the original trial was based on the 50
trial triplet years. Again, my colleague Dr Donnelly may wish
to enlarge on this. A number of assumptions were taken into account.
Those assumptions may, in fact, lead us to have useful data before
the end of the 50 triplet-year period or the 50 triplet-year period
may be truncated depending upon the incidence of cattle TB breakdowns
in our trial areas. So we suggested in that document that it may
be as early as 2002 that we have some useful hard data that can
be usefully translated into policy options. It is more likely
to be into 2004, but depending upon the strength of the data it
could be beyond that time. Do you wish to expand on that, Christl?
(Dr Donnelly) As is illustrated in the report, the
key thing is the number of breakdowns that are observed, and so
our ability to tell and to detect robustly a reduction in TB incidence
due to the interventionseither of a proactive or a reactive
cullingwill depend on what the baseline breakdown rate
is, the incidence in the control survey on the area. So if TB
increases in the underlying rate then that will mean we accumulate
data more quickly and are more quickly able to detect a difference;
whereas a somewhat conservative estimate would be a rate of eight
breakdowns per triplet per year, and that would bring us up to
the 50 triplet years, which would end at the end of 2005.
2. You said, in your second report, you expected
to begin analysis of the results from the trial in November this
year. I take it from your reply that has started. In your memorandum,
referring to MAFF, there are two areas where you expressed some
concern: one is about the delay in starting the road traffic accident
survey and the second one is the interruption in administering
the TB99 form. Are you confident that MAFF is putting sufficient
into those activities? Is it concentrating too heavily on the
(Professor Bourne) The trial is but one part of the
programme of work in place, as you well realise, and we have a
whole raft of research activity looking at cattle issues, particularly
cattle pathogenesis, a better understanding of disease in cattle
and having improved diagnostic tests. With respect to the RTA
that you mentioned, and also the application of TB99 on farms,
no we are not totally reassured and MAFF are aware of that. The
RTA is being delayed because of a number of issues, one being
health and safety and, latterly, of course, the swine fever outbreak
in East Anglia, which has taken a lot of MAFF staff away from
other activities including some of the TB post mortem activities.
RTA is now in place and we are relieved and reassured that the
thing actually is on the road, but we are concerned that its scope,
at present, is quite different to the scope that we envisaged
initially. We have told ministers we will accept the situation
as it stands, but we would wish to review the situation in January
when we expect the swine fever episode to come to an end, and
expect a more rigorous input from MAFF into the road traffic accident
survey. With respect to TB99 and the application of these on farms,
we again accepted that MAFF were in some difficulty with respect
to staff being transferred to swine fever duties. We were extremely
concerned, nonetheless, that TB99 was not being carried through
in trial areas. We now have the reassurance of MAFF that this
is being done. We understand there will be some delay in getting
these done outside the trial areas, but we are less concerned
about that. We have also expressed concern that even within trial
areas it is being done by animal health staff rather than veterinary
staff, and we sought reassurances on the quality of staff input
into TB99. We have received from MAFF those assurances.
3. Do these concerns amount to a, in your view,
serious compromise of what you are trying to do? Or are they a
sort of niggling detail?
(Professor Bourne) With respect to TB99 I would hope
it is a niggling detail, but with respect to the road traffic
accident survey, clearly, it would have been better if we had
had this in place two years ago and not now. It is the only handle
we have on getting some information on the prevalence of TB in
badgers outside trial areas. It could be, at the end of the day,
that the RTA proves to be a useless tool, but unless it is done
properly we will not know that. So we are more concerned about
the RTA at the moment. As I said, we insist on coming back to
review this in January next year, and we would look then to more
effort being directed to the RTA.
4. The Committee commented in its initial report
on the doubts there were about the statistical power of the trial.
The response that you gave dismissed those doubts. Would you like
to expand on why you feel that was right?
(Professor Bourne) I do not think it is true to say
we dismissed those doubts. We took it very, very seriously, and
continue to take it very seriously.
5. I think it was said to be based on a "total
misunderstanding" of the role of the power of calculation.
I must admit, I use dismissive language myself sometimes, and
that is the kind of thing I might say.
(Dr Donnelly) There have been people who raised concerns
about the statistical power and when we were last here we discussed
that and the Committee recommended that an independent statistical
auditor be appointed and review what was going on in the trial
from a statistical basis. That has since happened, he was appointed
in August of this year and he has provided a preliminary report
that has gone to MAFF. That was supportive of what we were doing.
The concerns that had been raised looked at the assumptions that
we have made underlying the power of calculations; that is the
assumption of a Poisson distribution for breakdowns. The Poisson
distribution is appropriate for a series of events where you are
dealing with incidence data, but it assumes that all breakdowns
are occurring independently once you have taken account of predictive
factors. Such as, if you had it you could account for badgers
in a particular area exposing two different farms to the same
potential risk. However, beyond things that you have accounted
for they are independent. We have acknowledged that that will
not be absolutely the case; any time you do power calculations
you have to make assumptions about things and they will not be
absolutely independent. I have looked at the distribution of breakdowns
in farmshow many farms had multiple breakdowns over a period
of years prior to enrolment in the trialand they were consistent
with the Poisson distribution. The other thing that relates to
the power to detect differences of certain magnitudes is the underlying
incidence rate. The 50 triplet years that we have talked about
is actually quite conservative in assuming that the underlying
breakdown rate will be eight breakdowns per triplet per year.
So to the extent that there is some non-independent that will
reduce a little bit the power that we have, the fact that we have
made this very conservative assumption about what the incident
rate is means we will actually have greater power.
6. What is your estimate of the power, based
on your current knowledge?
(Dr Donnelly) It depends on what sort of breakdown
rate you are actually looking at. If you look at
7. This is a bit critical, is it not? If it
is not going to produce strong enough results which give a clear
enough indication of the relationship, then people will feel that
they have wasted a huge number of badger lives and a large amount
of money to demonstrate something which may be possible but not
a strong enough relationship to establish policy. So it is critical
to get this rather tedious and complexwhich I cannot claim
to fully understandstatistical methodology right to ensure
that we are going to be guided by policy which is founded on science
and a proven incidence of this particular problem.
(Dr Donnelly) We considered as an example a 20 per
cent reduction in incidence and found that we have a 90 per cent
power, under the assumptions that are made to detect that at the
end of the 50 triplet year period, which would be the end of 2005.
8. Have you checked back on the information
you have gathered so far from the early data to check that estimate?
(Dr Donnelly) You mean since the trial has started?
(Dr Donnelly) I have done one interim analysis which
I was given blinded, so I did not actually know which treatments
were allocated to whichinformation from three triplets
that have had proactive culling prior to this. I would say the
data appeared to be consistent with the assumptions that we have
made, but I would also caution that that data is based on 3 triplets.
10. Also, presumably, if the triplets that you
drew data from were from traditional hot-spots, there is an argument
for saying that the data collected may not apply in other places
where infection has been much more recent.
(Dr Donnelly) We have taken into account in all our
selection of triplet areas the incidence in the past year and
the past three years, at the time the selection was made. So at
that point that is the best incidence data that we have, and that
gives us the best prediction of what will happen in future.
11. What is the minimum reduction in TB incidence
that you would consider it important to detect, and within what
time-scale in this trial?
(Dr Donnelly) It is extremely difficult to answer
given that we do not yet have the information, but we have talked
about doing an economic analysis and talking about that in the
broader sense of economic, taking into account a number of concerns,
to look at the cost-benefit analysis of doing a sort of culling
intervention. So if it turns out that a 20 per cent reduction
in breakdowns is possible and we are able to detect that, but
that it actually costs more to complete the intervention than
you would save by doing that, then a 20 per cent reduction would
not be sufficient to suggest that that was a sensible policy.
So as yet we cannot actually say, but there are going to be research
programmes looking at what the economic impacts are to help us
make that judgment.
12. You mentioned Professor Mollison's scrutiny
of the analysis and that he had given that to MAFF. Could you
say a little bit more on what he has said so far, or should we
ask the Minister that question?
(Professor Bourne) We have seen a draft copy of that
report and the full report, I believe, has gone to MAFF and it
would be appropriate for you to ask them about that. From our
perspective it is certainly reassuring that his assessment of
the statistical validity of the trial agrees with our own.
13. Are you prepared to make data available
to outside sources to validate your assumptions?
(Professor Bourne) You saw the document we prepared
for Professor Mollinson, which I believe is part of your papers.
14. Is that more widely available?
(Professor Bourne) I think it is in the public domain,
is it not, as indeed are all the papers related to this?
15. Not yet if it has been given to us, but
it may well be.
(Dr Donnelly) Our submission is available on the MAFF
16. Can I return to this point of the power
of the calculations and the results? We have had a memorandum
from a Dr Fiona Mathews, a Hodgkin Research Fellow at the Department
of Zoology in Oxford. She says that the original sample sizes
appear incorrect. "Rather than having a power of 90% to detect
a reduction in TB incidence of 20% within 5 years, the true power
is likely to be nearer to 50-60%. This means that if culling badgers
reduces cattle TB by up to 20% then there is a 40-50% chance that
it would not be detected. Even were the effect of the badger cull
greater, and cattle TB was reduced by as much as 25%, there would
still be less than an 80% chance that the trial would give a `positive'
result within 5 years." How do you react to that?
(Dr Donnelly) I think you were given the summary and
I have seen the paper; Fiona Mathews provided it to us, actually,
a year ago. So I have seen her concerns, and the concerns that
she has are the clustering of TB breakdowns of the non-independence
that we suggested and, also, variations between triplets in incidence.
She is taking concerns that are valid to the extent that we had
to consider non-independence, we had to consider clustering and
repeat breakdowns, and so there is a valid area for concern. We
have examined those, we have looked in detail at the data on past
incidence in the trial areas, and we found that the heterogeneity
that she is assuming to get those sort of reductions is not evident
in the data. What she is saying is if those sorts of difference
appear as the trial went on, then the power would be reduced.
We cannot say absolutely what will happen in the datawhat
will arisebecause we have not seen the data yet. However,
on all the data that we have looked at so far, including the past,
we have no reason to think that we are getting that sort of level
of non-independence and that it would have such a reduction on
our power. Those are the exact issues that we discussed with them
17. So she is wrong?
(Dr Donnelly) She has taken a very extreme view, which
I do not think is representative of what is going on in the trial.
18. She is, in fact, saying that your original
calculations on the power of the study were incorrect and that
you really need more triplets.
(Dr Donnelly) Well, more triplets or more triplet
years. Yes, she is saying that.
19. You are defending the correctness of your
original calculation90 per cent power.
(Dr Donnelly) Yes.