SMART ACQUISITION: LITTLE PROGRESS IN MEASURING SUCCESS
38. The Department has not yet set targets for its
performance measures for the assessment phase and does not have
a measure to show whether risk is reduced sufficiently to enable
a sound decision to be taken at Main Gate. Asked why it had taken
so long to set these performance measures and when they would
be fully in place, the Department said that setting the measures
was difficult, and that the measures agreed would have to be used
for all assessment phase projects, not just those in the Major
Projects Report. However, the Department promised it would have
the measures in place by 1st April 2001.
39. Following the changes to the approval process
associated with Smart Acquisition approvals the Department approves
the cost and in-service date of an equipment which it is 90 percent
confident of achieving, while basing its plans on a lower/earlier
figure which it is 50 per cent confident of achieving. Timescale
and cost performance is measured in the Major Projects Report
by comparing the current forecast in-service date or cost underpinning
plans (the 50 percent estimate) with the latest acceptable in-service
date or highest cost set at approval (the 90 percent estimate).
This means that individual projects approved under >Smart'
procedures will be forecast to enter service earlier or cost less
than their approved in-service date or procurement cost unless
all of the risks allowed for in the 90 per cent approval materialise.
40. The differences between the 50 percent and 90
percent figures can be very large, for example, the Multi-Role
Armoured Vehicle is reporting a 31-month negative in-service date
variation. In this case, the Department said it was unfortunate
that one of the first projects to appear under the new regime
was a two-nation collaborative programme which might soon become
a three nation programme if the Netherlands joined. This uncertainty
had increased risk, and that there were a number of other international
factors which would bedevil the programme through development.
If these uncertainties were added up, the 31 months did not look
like a generous allowance of time.
41. Asked how large, on average, it expected the
risk provisions for time and cost to be at Main Gate, the Department
suggested that the absolute differences would be variable between
projects. The fact that one project approved under ASmart@
procedures, the Seawolf Midlife Update, had already breached its
90 percent in-service date approval by 3 months showed it was
not being institutionally generous to itself.
42. All of the projects covered in the Report were
conceived before Smart Procurement, and the Department suggested
that it was still early to expect to see improvements in timescale
performance from the application of Smart Procurement principles
reflected in the Major Projects Report. Only when projects with
significant delays such as the C-130J begin to pass out of the
Report, and new >Smart'
projects entered the Report, would the average delay decrease.
The Department was confident that this effect would start to become
apparent from the Major Projects Report 2002.
43. The Department is not clear what level of risk
it expects will remain in projects at Main Gate and how this will
be measured. A much clearer definition of the acceptable level
of risk at Main Gate is needed and the Department must give priority
to defining this and how to measure it.
44. Under Smart Acquisition, the Department bases
its Main Gate Approval on the cost and in-service date of an equipment
which it is 90 percent confident of achieving. We note that the
SeaWolf Mid-Life Update project has already exceeded the in-service
date which the Department was 90 percent certain of achieving
by some three months. This does not bode well for the future.
45. The Department does not expect there to be big
improvements in the timescale performance reported in the Major
Projects Report until 2002, some four years after Smart Acquisition
was first introduced. Whilst the timescale performance reported
in the Major Projects Report will continue to reflect pre-Smart
Acquisition projects for some years, we will expect to see signs
of improvement as new Asmart@
projects come through.
46 Qs 202B203 Back
Report (HC 970 (1999B2000)) para 1.13 Back
48 Q204 Back
49 Q205 Back