Annex 1
A chronology of the FADEC problems from the start
of the development in the early 1980s to the position today may
help the Committee. Documents to support any of the facts reported
in the chronology can be made available to the Committee.
1984-87
Delivery of the FADEC was promised by the main
contractor, [Textron] Lycoming, within 23 months of the date of
contract which was signed in January 1986. Flight test validation
was due to start by the end of September 1987. In fact the first
flight tests of a FADEC started in 1989 and were abandoned when
an MoD Chinook was badly damaged by a sudden and unexpected FADEC-induced
acceleration of the engines [see 1989] The First FADEC-equipped
Chinooks were not approved for operations until November 1993,
and then without reservations that were expressed by the Ministry's
own airworthiness assessors at Boscombe Down and by the MoD's
Chinook Project Manager in the Procurement Executive.
Problems in the relationship between the MoD/RAF
and the software suppliers date back to the early days of software
development. In 1984, the RAF's Chinook Liaison Office wrote that
it considered [Textron] Lycoming was being secretive. The RAF
noted that it was unable to deal directly with the FADEC's software
contractor. On all queries about FADEC, the RAF had to contact
the FADEC's prime contractor [Textron] Lycoming in the USA—althought
he software was being written in Britain by Textron's subcontractor
Hawker Siddeley Dynamics Engineering.
1988
RAF-monitored tests on the FADEC highlighted
"numerous discrepancies".
1989
The first major series of "flight"
tests on a FADEC, as installed on a Chinook, ended in a serious
accident at Boeing's Flight Test Facility at Wilmington, Delaware,
then an MoD Chinook was almost destroyed on the ground by an engine
overspeed. The accident led to the MoD taking legal action against
the manufacturer over what it called a "fundamental flaw"
in the design of the FADEC. In the accident, the FADEC allowed
the engines and the rotors (which cannot be disengaged from the
engines) to accelerate out of control of the pilots—a so-called
engine runaway. At the MoD's request, the FADEC was modified but
there was not a rewrite of the software and flaws remained at
the time of the crash on the Mull of Kintyre in June 1994 (see
1994).
1990
The British Defence Staff Washington (based
at the British Embassy in Massachusetts Avenue) wrote a letter
to the Vice president Military Engine Programs at Textron Lycoming
in Stratford, CT, claiming $5,815,350 related to losses on the
MoD Chinook that was "severely damaged" in an engine
overspeed on 20 January 1989. "Following a careful investigation
of the overrun incident and its causes, we have concluded that
the overrun and ensuring financial losses were the direct result
of Textron's failure to meet the Contract's requirements, and
the failure of Textron (and its subcontractors for whose work
Textron is responsible) to use due care in the design and testing
of FADEC".
1993
The British Government, having, asserted a claim
against Boeing, released Boeing from all claims related to the
overspeed incident on 3 September for the consideration of $500,000.
The MoD said that it was "entirely Textron's fault that the
damage was done". Boeing was not asked to accept liability
nor did it offer to accept such liability.
Also in 1993, the disagreements between Textron
Lycoming and the A&AEE at Boscombe Down over the quality of
the FADEC software, were one reason that EDS-Scicon, an independent
defence contractor, was brought in to comment on the software.
EDS-Scicon agreed with the A&AEE. EDS-Scicon abandoned a review
of the code after examining 18 per cent of the code and finding
485 anomalies. EDS-Scicon suggested a re-write of the software.
Textron Lycoming rejected this suggestion as unnecessary. EDS-Scicon
had found 56 "category one" anomalies in the software.
EDS-Scicon said that safety-critical code should ideally have
no anomalies in this category. There were a further 193 anomalies
in category two, when there should have been very few indeed,
according to EDS-Scicon. These 249 category one and two anomalies
were found in only 18 per cent of the code. EDS-Scicon had abandoned
its review at this stage because of the density of anomalies it
found.
At this time operational commanders had to meet
a high demand for Chinooks but the number of the helicopters available
was, at one point, down to 40 per cent.
The MoD and RAF took advice other than Boscombe
Down's and put the Chinook into service. The official justification
for over-ruling A&AEE was given to the Defence Committee in
1998. The MoD said it took into account the fact that there had
no safety-critical FADEC incidents in the US Army. It also took
into account the fact that the FADEC was not safety critical because
not even the engines were safety critical. Without its engines
the Chinook could glide safely to earth. Indeed the Defence Committee
was told in 1998 that Boeing and the MoD did not consider the
FADEC to be safety critical. "Boeing did not consider the
FADEC to be flight safety critical because the engines on the
Chinook are not considered to be safety critical," said an
MoD report to the Defence Committee (page 33—fourth report
dated 13 May 1998).
However, Boeing did classify the FADEC as safety
critical, and the engines too. The MoD also regarded the FADEC
as safety critical. An MoD document prepared for the arbitration
proceedings against Textron Lycoming said: "The FADEC as
designed by Lycoming . . . had few safety features that did not
depend on the software. The software was truly critical in maintaining
safe flight".
And although it was true that the US Army had
not had any FADEC incidents, not a single operational Chinook
was flying in the US with an RAF version of the FADEC at this
time. So it was true that there had not been a safety-critical
FADEC incident in the US Army. But there had been a safety-critical
FADEC incident involving an MoD Chinook—in 1989 at the Boeing
Flight Test Facility.
So the MoD was approving the Chinook partly
on the basis that there had been no serious FADEC problems in
the US—yet there had been serious FADEC problems with an
MoD Chinook.
Aware that they were putting into operational
service a helicopter which had a FADEC with known deficiencies,
the MoD and the RAF approved the Chinook for release in November
1993. Partly because of the A&AEE's concerns about FADEC,
a weight restriction was imposed in case FADEC caused one engine
to fail. This restriction did not, however, take account of a
more serious risk—a sudden acceleration of the engines that
was not commanded by the pilots. It was an uncommanded acceleration
of the engines that caused the accident in 1989.
7 March 1994
The unexpected flameout of an engine on a FADEC-equipped
Chinook Mk2 led to the A&AEE at Boscombe Down suspending trials
flying. The suspension was lifted on 20 April but resumed again
on 1 June 1994.
April 1994 onwards
Pilots on Chinooks fitted with FADEC discovered
that the Chinook's engines sometimes accelerated or decelerated
suddenly and without warning. A Squadron Leader at the main UK
Chinook depot, RAF Odiham, later told an RAF Board of Inquiry:
"The unforeseen malfunctions on the Chinook HC2 of a flight
critical nature have mainly been associated with the engines control
system FADEC. They have resulted in undemanded engine shutdown,
engine run-up, spurious engine failure captions, [warnings in
the cockpit and misleading and confusing cockpit engine indications".
The two pilots of the Chinook HC2 who died in
the crash on the Mull were among the pilots who had expressed
concern about flying the upgraded aircraft. The RAF Board of Inquiry
was told that the two pilots were "worried" about the
"uncertainty of how the aircraft's Fully Automatic Digital
Engine Control system would perform during operational sorties
in Northern Ireland and what sort of emergencies or situations
the present amount of spurious and unexplained incidents would
lead to".
On 21 April 1994, the advice of the MoD Procurement
Executive was that it was "important to understand and take
full account of A&AEE's views".
26 May 1994
In the latest of a series of incidents involving
FADEC, the pilots of Chinook ZD576 (the aircraft that crashed
on the Mull of Kintyre) reported more FADEC-related problems.
During a flight, it was found that various warnings were given
in the cockpit, including a "master" warning, indicating
a possible engine overspeed, and a No 2 engine failure notification.
The pilots diverted to Luton. The pilots reported that the engine
failure warning took 10 seconds to extinguish.
1 June 1994 (a day before the crash on the Mull
of Kintyre)
The A&AEE, for the second time in 1994,
ceased trials flying of the Chinook fitted with FADEC because
Textron Lycoming had not given satisfactory answers to questions
raised by the A&AEE about the system.
The official dispensation to trials pilots not
fly Chinooks fitted with FADEC did not apply to operational pilots.
Flt Lt Jonathon Tapper, one of the pilots of
the Chinook HC2 that crashed on the Mull requested permission
from Squadron Leader David Prowse for the use of a Chinook HC1
(not fitted with FADEC). This was because of the icing and other
restrictions that were imposed on the Chinook HC2 as a result
of the FADEC problems. The request was refused. Flt Lt Tapper
and his co-pilot Flt Lt Rick Cook could have refused to fly Chinook
ZD576, but this could have been seen as insubordination.
2 June 1994 (the day of the crash on the Mull
of Kintyre)
Chinook ZD576 flown by Flt Lts Tapper and Cook
who were sufficiently highly regarded and experienced to be given
the status of Special Forces pilots, took off from Northern Ireland
for Scotland and crashed on the Mull of Kintyre. Everything appears
to have been normal until the last half-minute or so before impact.
What happened then is the subject of dispute. Critics of the decision
to blame the pilots contend that, as there is no confirmed evidence
of what happened in the last moments of flight, nobody will ever
know if the aircraft was under control. The MoD contends that
as there is no evidence of a malfunction, the aircraft was undoubtedly
under control. The MoD also contends that the pilots saw the bad
weather on the top of the Mull, and should have transferred to
instrument flights rules, climbed to a safe altitude, slowed down,
turned away or turned back. As they did none of these things,
they were grossly negligent.
Dozens of aviation specialists, including RAF
officers, former and serving, have contended that the issue is
not whether the pilots flew at speed into the Mull, the top of
which was covered in cloud—it is obvious that they did—but
whether the pilots flew voluntarily or compulsorily into the Mull.
Computer Weekly does not believe that the answer to this
question will ever be known.
December 1994
The MoD completed its reports for the American
Arbitration Association Commercial Arbitration Tribunal, in the
case against Textron Lycoming. The reports said that:
— Faulty logic in the design of the
FADEC software was the immediate cause of the accident in 1989.
— The FADEC was not airworthy.
— The software was not adequately documented.
— The Failure Modes Effects and Criticality
Analysis was seriously incomplete.
— The Subsystem Hazard Analysis failed
to cover possible failures that were critically important.
— Development tests on the FADEC hardware
were inadequate.
— The software was not adequately verified.
— [Textron] Lycoming failed to comply
with DO-178A (an international avionics standard).
— [Textron] Lycoming failed to meet
the comparable design documentation requirements of JSP188 (an
international documentation standard).
— [Textron] Lycoming did not adequately
respond to warnings of design flaws.
— [Textron] Lycoming failed to supervise
work adequately.
— Lycoming adopted an unnecessarily
high-risk design strategy.
— Lycoming's failure to maintain schedule
may have caused it to take short cuts with safety.
As these reports were never published or shown
to the RAF Board of Inquiry or the Scottish Fatal Accident Inquiry,
it was not realised that FADEC was capable of causing a potentially
fatal accident and indeed had caused a serious accident in 1989.
1995
The RAF Board of Inquiry report into the crash
on the Mull of Kintyre was published. Quotes from the report are
as follows:
— In considering the cause of the accident
the Board were severely hampered by the lack of a Cockpit Voice
Recorder and an Accident Data Recorder
— Operationally both pilots [of ZD576]
had a very stable and constructive attitude towards their flying.
The concerns they had with reference to the Chinook HC . . . did
worry both pilots in two ways. First was the uncertainty of how
the aircraft's Fully Automatic Digital Engine Control System would
perform during operational sorties in Northern Ireland and what
sort of emergencies or situations the present amount of spurious
and unexplained incidents would lead to".
— The Board considered engine control
system malfunctions and it is particularly relevant to note that
at this stage of the Chinook Mk2's service, spurious Engine Failure
captions [warnings], lasting on average seven-eight seconds, were
an increasingly frequent occurrence. These are now well understood
but at the time they were not. Had such an indication occurred
it would have caused the crew considerable concern particularly
as they were over water with no obvious for an emergency landing.
Such a warning would also have required an urgent and very careful
check of engine instruments and Flight Reference Cards (the engineering
manuals).
— The Board could find no evidence
that Flt Lt Tapper had not approached and prepared for the sortie
in anything other than a thorough and professional manner. The
Board was unable to determine the sequence of events leading up
to the accident...it would be incorrect to criticise him (Tapper)
for human failings based on the available evidence".
— The Board reviewed the technical
malfunctions and air incidents which had occurred with the Chinook
HC2 in RAF service and considered whether they could have played
a part in the accident. The Chinook HC2 had experienced a number
of unforeseen malfunctions, mainly associated with the engine
control system, including undemanded engine shutdown, engine run-up,
spurious engine failure captions, and misleading and confusing
cockpit indications. The board found no evidence that any of these
malfunctions had occurred on Chinook ZD576's final flight. Nevertheless
an unforeseen technical malfunction of the type being experienced
on the Chinook HC2, which would not necessarily have left any
physical evidence, remained a possibility and could not be discounted.
— In considering the available technical
information, the Board concluded that technical failure was unlikely
to have been a direct cause of the accident. However, given the
large number of unexplained technical occurrences on the Chinook
HC2 since its introduction, the Board considered it possible that
a technical malfunction could have provided a distraction to the
crew".
— The Board concluded that distraction
by a technical malfunction could have been a contributory cause
of the accident."
The inquiry report said that although the pilots
may have chosen an inappropriate rate of climb to fly over the
Mull, there was insufficient evidence to blame them. A different
view was taken, however, by two air marshals who reviewed the
Board's findings: They judged the pilots to have been negligent
to a gross degree. One of the air marshals said:
"In my judgement none of the possible factors
and scenarios are so strong that they would have been likely to
prevent such an experienced crew from maintaining safe flight.
Therefore, unlike the Board and the Officer Commanding RAF Odiham,
I reluctantly conclude that the actions of the crew were the direct
cause of the accident."
The view of the air marshals assumed that no
evidence of technical malfunction meant pilot error.
In the years since the crash, more information
has come to light regarding the criticality of the "FADEC"
engine control software". Various statements issued by the
MoD show that officials appear not to have been aware of the extent
to which the near destruction of a Chinook in 1989 was caused
by a design flaw in the FADEC software.
Indeed the Defence Committee was told repeatedly
that the accident in 1989 was caused by negligent testing procedures
and was nothing to do with the software. Computer Weekly
has MoD documents showing that the Ministry believed that the
opposite is the case. In its litigation over the 1989 accident,
the Ministry's case exonerated the testing procedures and found
that the cause of the accident was a faulty design of the software.
Autumn 1995
The MoD won its case against Textron Lycoming
and was awarded about $3 million in damages.
1996
A three-week Fatal Accident Inquiry in Scotland
found that there was insufficient evidence to blame the pilots.
The inquiry was not told about the litigation over FADEC, the
EDS-Scicon report, or any of the memos or reports from A&AEE
and the MoD Procurement Executive regarding the FADEC.
1997
At 1,100 feet, during a normal cruise stage
of flight, a US Army Chinook suffered a jam of the controls and
turned upside down. With the pilots trying to free the controls,
there was no time to make an emergency call. At 250ft about the
ground the controls inexplicably freed themselves and the helicopter
landed safety. No evidence of technical malfunction was found
and the technology of computer simulations was not sufficiently
advanced enough to replicate the circumstances of the incident.
Had the crew died in a crash they may have been
blamed, as there was no evidence of a serious technical malfunction.
1998
The Defence Committee was told that:
— The A&AEE's concerns did not
relate to safety but only to the fact that they could not read
the FADEC software and therefore could not verify it. The Committee
was not told that Boscombe Down had read the software and had
found it unacceptable.
— The software was not safety critical.
Some months after the hearing officials confirmed in writing that
Boeing had considered the software to be safety critical.
— That the litigation against Textron
was "nothing to do with software" only negligent testing
procedures.
— That FADEC problems prior to the
crash on the Mull were "trivial" and related only to
"soft" faults.
— That the A&AEE's concern about
the software was because it was trying to test the code using
a method of testing software—static code analysis—that
was not widely used, apart from in the nuclear industry (although
the MoD's own standard for safety related software 00-55, dated
1994, recommended static code analysis).
— That the US Army was happy with its
Chinook fleet (although the US Army did not have the same FADEC
version)
— That there had never been a FADEC-caused
accident (although the accident in 1989 was caused by a faulty
design of FADEC).
The Defence Committee concluded that "although
the FADEC system has received much attention in the media and
elsewhere it is far from being the main source of reported faults.
Indeed engine problems more generally were only 25th on the MoD's
list and of the 11,000 faults reported across all of the aircraft's
systems, FADEC failures represented less than 0.2 per cent".
The Committee was persuaded that the "failure
of Boscombe Down to give final approval to the FADEC software
. . . is a management failure".
1998-99
In defending the decision of the two air marshals,
the MoD went well beyond what was said at the RAF Board of Inquiry.
For example, officials have made three factually disparate statements
on how, when and at what point on the approach to the Mull the
pilots were negligent.
2000
A report for the national Transportation Safety
Board in the US confirms that no evidence of technical malfunction
does not mean no technical malfunction.
| 
