Examination of Witnesses (Questions 1-19)|
WALMSLEY KCB AND
MONDAY 10 DECEMBER 2001
1. Order, order. Welcome to the Committee of
Public Accounts and welcome Sir Robert back to the Committee and
Vice Admiral Sir Jeremy Blackham who are going to give evidence
today on the Major Projects Report 2001. We are delighted to see
the progress you are making with Smart Acquisition. As you know,
this has long been a particular interest of this Committee. Can
you tell us some of the key initiatives you are taking to ensure
that Smart is succeeding and how are you going to build on the
progress you have already made?
(Sir Robert Walmsley) We start with a
structural one of forming Integrated Project Teams. That has been
right at the heart of Smart Acquisition. Secondly, we decided
who our customer was and I am delighted to say he is sitting on
my left. That produces a clarity as to who is entitled to commission
articles from the Defence Procurement Agency and that has rippled
out through into an approach which is called a through-life approach,
which makes sure that we are far better constituted to take into
account costs which arise after the capital acquisition. Beyond
that I would just mention two things: one is better project management,
which you should never cease to strive for and new techniques
are always coming through and we try to learn them and apply them;
learn from each other as well as learn from outside. The final
point is to recognise that the single most dominant cause of delays
is technical difficulties. We need to take a new approach to the
rigorousness with which we assess the technical risk remaining
in a project at the main investment point and the tool we have
chosen to do that is called technology readiness level assessment.
2. May I ask you a bit about slippage and in
particular Figure 9, which is on page 12? Can you tell us a bit
more what the Department can do to address these causes of slippage
and when the measures you have in mind will take effect to slow
the rate of slippage. Slippage is still taking place on all projects,
not just some, but it is slowing. Perhaps you could tell us a
bit more about that.
(Sir Robert Walmsley) It only slipped on four projects
this year, but that does not mean to say there is not a terrible
history sitting there. If you look at Figure 9 you can see the
top line says that technical factors, that is 331 months, sit
at probably slightly more than half the total delay. I explained
that I think that is because we take on too much technical risk
in some of our projects. However, we are not the only ones, because
there is a table further on in the report which shows that the
Department of Defense in the United States are on the same concern
as us and they had an assessment by an outside body who concluded
that they were entering projects with technology at an insufficiently
mature level. I am quite sure we need to do something about that.
That is Table 12 on page 18. So that shows you that we are not
alone in starting off on projects with technologies which are
not sufficiently mature. We have introduced on some projects and
will be introducing increasingly as we approve new projects, this
concept of technology readiness levels. What that is, is a ladder
which runs from one to nine and you are at level one when you
have had an idea and you are somewhere about level two when you
have shown in a laboratory that it can work. You work through
trying to integrate that as part of a system which might take
you half way up the ladder. If you had flight tested it, for example,
you might be at technology readiness level six and if you had
proved it in a system, you are at technology readiness level eight.
Somewhere between six and eight is a pre-condition for embarking
on a major new technology for a new project. That is the idea.
It forces you to gather objective evidence as to whether the technology
is mature enough, not just promise it because you think it is
a good idea.
3. If we look to paragraph 2.5 we see that you
are not currently measuring risk reduction during the assessment
phase. Can you tell us a bit more about the amount of time or
whether you indeed spend the right amount of time and money on
projects in assessment right now?
(Sir Robert Walmsley) I think we do set the level
about right, but I am concerned, as the report is, that we do
not yet have an objective measure as to whether the assessment
phase has accomplished its objectives. Most obviously, if we cannot
make a decision to proceed through Main Gate, then it is reasonable
to conclude that the assessment phase did not do enough. In a
sense that is not quite good enough because the real proof of
the pudding is whether the subsequent activity, that is to say
after Main Gate, actually delivered a project on time and delivered
performance within budget. A second point, obvious but still worth
mentioning, is that the assessment phase should itself be conducted
to budget and to time. If there are any exceptions to that, then
they need to be explained. It does not mean they should be prohibited,
but that if you discover something during assessment you should
investigate it rather than pretend it did not happen in order
to avoid a cost increase. The third point is the most sophisticated
one. I shall try to explain it. Three-point estimates are a fundamental
tool and give you an idea of the uncertainty remaining in a project:
what is the lowest cost, that is a 10 per cent confidence level,
the expected cost, which is a 50 per cent confidence level and
the highest cost, a 90 per cent confidence level. It is quite
logical to me that when you start assessment there should be quite
a big spread there. By the time you get to Main Gate, that spread
should be much narrower and the narrowing needs to be supported
by objective evidence.
4. May I ask about three-point estimates? We
had been promised that you were going to have them for all projects.
That has still not materialised. Can you explain why?
(Sir Robert Walmsley) They have not materialised in
the report because some of them were just at a critical point
as we came through the back end of last year, through towards
the datum date for this report, which was 31 March. I updated
the Committee with one or two further three-point estimates in
the summer. I am happy to say that as I sit here today I do have
three-point estimates for all the projects in the assessment phase
which are going to go ahead. We are completely reviewing the procurement
strategy on TRACER following the United States' decision to pull
out. I just want to make it clear that I am committed to three-point
estimates, I have them here on a piece of paper sitting opposite
the table which shows them incomplete and they are now complete.
5. Let us go into that in a bit more detail
and try to tie you down with a few questions. Presumably from
what you have told me you agree that consequent on what the Committee
of Public Accounts has advised in the past, all projects do have
full three-point estimates at initial gate. Do you accept that?
(Sir Robert Walmsley) I absolutely agree with that.
6. How often do you think these estimates should
be reviewed between Initial Gate and Main Gate?
(Sir Robert Walmsley) A lot depends on the project.
It seems to me that it is a fundamental management tool as to
whether assessment is delivering the objectives. Every time you
take a deliverable from a contractor, you should see whether that
has had any impact on changing the spread of the three-point estimate.
7. From what you have said I presume the answer
to this is no. Are there any circumstances in which projects should
not have three-point estimates?
(Sir Robert Walmsley) There are no circumstances in
which they should not have estimates. There are many circumstances
in which those estimates should not be disclosed to the potential
8. Should projects be using these estimates
in their management of projects? Will all projects in the Major
Projects Report 2002 have reliable three-point risk estimates
for cost and time? I am asking you for a commitment now which
I shall be interested to hear whether you can give. We will hold
you to it.
(Sir Robert Walmsley) I understand that. I shall just
pull one word out from that question, to which I shall return,
and say yes, unequivocal yes. The word I should like to take out
is "reliable". You qualified it and said "reliable
three-point risk estimates". Soundly based yes; for me to
guarantee the reliability would be guaranteeing that what I predict
will happen in the future will in fact happen. They will be properly
based, they will be analytical, they are not holding a wet finger
up in the air. They will be soundly based and yes is the answer.
9. But not necessarily reliable.
(Sir Robert Walmsley) "Reliable" is asking
me to guarantee the future.
10. That is fair enough. A lot of claims have
been made that we are going to be saving £2.4 billion because
of Smart Acquisition, but when we look at Figure 16 this shows
that 70 per cent of Smart cost reductions are expected to arise
on only 12 projects. We have to ask you why Smart Acquisition
is affecting projects so disproportionately?
(Sir Robert Walmsley) Smart Acquisition is a technique
for looking to see whether there is a better way of doing things.
What that means is that when we look at the cost estimate for
a programme, we try to see whether there is a cleverer way, a
smarter way of discharging a programme which will produce savings.
Most of the projects, I hope, have very tight estimates; that
is how people prepare estimates and therefore the scope, the assumption
that there must be scope for saving money is something I want
to be quite cautious about. Below and beyond this table in the
great population of projects there are other savings which could
properly be characterised as Smart. I pick an example which occurred
to me as I read the table, the increased use of just-in-time delivery
of sub-contracted supplies means that the value of the asset under
construction, particularly in the case of a ship will not build
up so early during its build period. What that means is that this
new concept of paying interest on capital or assets in the course
of construction to represent the opportunity cost of seeking to
invest in something which is not delivering to the front line,
if we deliver bits later, then there will be a smaller total interest
on capital charge. The Astute submarine through a combination
of later delivery and a number of other measures associated with
optimising the construction of the submarine, will save in this
LTC about £90 million. That has not been characterised as
a Smart saving because the people doing it thought it was normal
business. Beyond the list here there is a great number of them.
I just start by saying that the huge saving attributed to Skynet
5 right in the first line is as a result of our confidence that
we can execute the programme through the private finance initiative
or public/private partnership. That has really compressed the
timetable for delivering a satellite and has resulted in this
11. A last question by way of light relief.
Sting Ray. You will see what is said in Box 1 about Sting Ray.
This is basically to try to stop you blowing yourself up if you
bounce the thing on the deck. What are you doing to minimise the
risks associated with handling Sting Ray?
(Sir Robert Walmsley) Sting Ray is handled very carefully
in frigates and there are set procedures for moving it on the
flight deck at night and a number of other issues related to the
arming of helicopters with Sting Ray torpedoes. The first thing
we do is make all those who are responsible for handling Sting
Ray aware of it. I have to say that the risk is absolutely minute
and proper handling can eliminate the risk. That does not mean
we should not be pursuing insensitive munitions as a subject.
12. You are having to work on this despite the
fact that the risk is absolutely minute because of Health &
Safety at work and all the things which people worry about nowadays
which they did not worry about to such an extent when both of
you joined the Navy. How much is this all going to cost in making
sure that Sting Ray cannot blow up if you drop it on the deck?
(Sir Robert Walmsley) We are doing a study now to
establish what it would cost to put in new design warheads and
when we have done the study we shall know the answer to that.
If you ask me to hazard a guess, and we were trying to do it across
the total number of torpedoes involved, it would probably be in
the region of £50 million, but that is just an informed estimate,
informed in the sense that I am experienced with torpedoes. It
is not informed by the results of the study. I just wanted to
put it in context for the Committee.
13. But the risk of this ever actually happening
(Sir Robert Walmsley) It is infinitesimal.
14. But you are now having to spend £50
million on tackling the problem.
(Sir Robert Walmsley) You are leading me by the nose
there. I did not say the £50 million was an established figure.
I wanted to do that to put it in context. There are risks associated
with fire, which we believe can be absolutely managed away, but
nevertheless we have a duty of care to provide a weapon which
is safe under all conceivable circumstances. Because that is now
possible, these are new warhead design techniques and it is now
possible to fix it, so we wanted to investigate the cost of doing
15. You have my sympathy. If there is a job
which is impossible, it is probably yours. On the one hand you
have a client who wants the best technology available and sometimes
not available but in someone's head and you then have to deliver
this by dealing with a set of contractors who really have all
the power over you and you pay up what they want and you bear
the risk. I should like to meet whoever called this Smart Procurement,
because it is procurement but not very smart at times, is it?
Your Department's strategic goal is to deliver 90 per cent of
projects with approval, to time, cost and technical performance.
Who picked the 90 per cent? Why not 80 per cent or 85 per cent?
Normally 100 per cent is picked. Why is it going to take until
2005 to deliver this, because this is within your own hands. Why?
(Sir Robert Walmsley) I would just make the point
that the contractors do not have power over us. We have power
over them because money is a tool: they want our money and in
competition we have a very good handle over them. Even when there
is no competition I believe you have a good possibility to exert
the power of a normal commercial deal on them. I should like to
come to the particular question. I said in answer to a point from
the Chairman that we assess cost on this three-point estimating
on the basis of a 10 per cent estimate, a 50 per cent estimate
and a 90 per cent estimate. It is clear that there is always a
remote possibility that something extraordinary could happen which
would reduce the cost, or awful could happen and could knock the
price beyond. In an analytical sense it simply makes sense to
try to do three-point estimating on a 10/50/90 basis rather than
0/50/100. If you now imagine there is a snapshot in time where
some projects have just been approved, when my guess is to say
that they would be perfect, it would be very, very unlikely that
a project approved yesterday would have bust its limits, as opposed
to another project which was just about to reach its entry into
service date, in which case you would expect 90 per cent of projects
still to be within tolerance, because that is what a 90 per cent
confidence limit does, you can see you will have a population
where some are perfect and the worst are at 90. What that means
is that on average they will be exhibiting the figures for about
95 per cent. It just so happens and you will recognise that from
this report, that there is an enormous number of legacy projects
which simply will not have worked their way out of the system
by 2005 and the best we can do, provided every single project
stays within limits is, having 90 per cent of our project within
approval by 2005 and the long-term goal is to get to 95 per cent.
16. I notice during the assessment phase of
this system the Committee took evidence last year and the Department
guaranteed to have performance measures in place by 1 April 2001.
I may have missed something but are they in place at the present
(Sir Robert Walmsley) We do have the performance measures
in place which I explained to the Chairman. We expect to discharge
the assessment phase to time and cost but we do not prohibit breaking
out of that if we find the reason to do that. I have also explained
the approach to narrowing three-point estimates and I have confirmed
that they are all now available. I am beginning to see the glimmer
of a system whereby I do not expect the spread on cost estimates
to be any more than 10 per cent at the Main Gate approval, but
we have not firmed up that number yet. I am really homing in on
what the National Audit Office asked for, which is a quantified
system for demonstrating the benefits of assessment. We are going
to use the narrowing of the spread of the three-point estimates
as one of the contributors to that.
Mr Jenkins: I have individual questions on individual
projects which I would have put to you, but I think the answers
I am going to get are going to be to justify the slippage, costs,
etcetera within this envelope of working within new frontiers,
pushing cutting edge technologies, going down that road, so I
shall take no more of your time.
17. How do you achieve the balance of eliminating
the risk from technology and also encouraging technological innovation?
Defence procurement has been a major source of technological innovation
in society. How do you achieve that balance?
(Sir Robert Walmsley) We have a very substantial technological
competence within the Department and I am including in that QinetiQ.
One of the reasons Admiral Blackham commissions a very substantial
applied research programme is to make sure that we have a very
good understanding of the technical possibilities. That does not
mean that they have to invent it themselves. It means the experts
in QinetiQ have to be aware, as are many of my own engineers,
of what is today technically in the pipeline and then think about
whether it is a reasonable thing to pursue it. The joint responsibility
of Jeremy Blackham and myself is to make sure that we spend enough
money during the assessment phase to give a rational rather than
just an enthusiast's opinion about the likelihood of that technology
delivering on time to cost and to performance. I think I explained
before that we need to discipline ourselves to take a more objective
view. There is no shortage of ideas. The tendency seems to be
that we take on board things which are frankly going to be tougher
in the reality than we had hoped when we had the initial advice.
I have to say some of that advice includes "promises"
from industry and it is only when they are contracted promises
that they get into trouble.
18. There is no danger that if you move to these
technology readiness levels you are going to over-compensate and
only go with technology which is completely tried and tested.
(Sir Robert Walmsley) I very much hope not, but I
do recognise that is a risk. You could solve an awful lot of my
problems, but not Admiral Blackham's, by just saying that is too
risky. One of the ways round that is that we are aware of our
competitors both industrially and in a defence sense and we are
not going to let British defence equipment fall behind our competitors
either industrially or in defence capability terms.
19. That is good to hear. May I ask you about
cost overrun? I take on board what the report says about the improvements
being made on this. I am still right in saying, am I not, that
according to the report of your 20 post Main Gate projects you
are running at £2.6 billion cost overrun? I am new to this
field. Can you set that in some sort of historical framework?
Is it normal for the armed forces procurement to be running at
such a high level of overrun.
(Sir Robert Walmsley) The idea that it is normal would
be intolerable to me. It happens to be a little bit less than
it used to be. There are some reports which indicate that in other
countries the average annual cost increase for defence procurement
projects is around the figure of two per cent. So we are doing
a bit better than that. If you look at other high technology projects,
Jubilee Line, computer systems, Channel Tunnel, civil aviation
projects, it seems to me we are not operating out of context with
high technology projects or indeed perhaps we are. Perhaps we
are doing a little better than some of those. I do not want to
give anybody in the Committee the sense that I am complacent.
Every single pound of those £2.6 billion is a pound we cannot
spend on other requirements and we should fix it.