Be earnest—or drift into failure

4690

An unsettling incident in a commercial turboprop illustrates the complex genesis of system failure.

In the dog days of 2013, between Christmas and New Year, a Cessna 441 Conquest was, unlike most of the country, at work. It was doing an aerial survey, but it never completed the job. The oil pressure gauge for the right engine showed minor fluctuations, which the observant flight crew noticed. After five minutes, these fluctuations increased to the point where the needle was entering the yellow band and the oil-pressure warning light was coming on. The pilots shut down the right-side engine, a Honeywell TPE-331s turboprop and made a pan-pan call. A wind change at the flight’s origin airfield meant the aircraft diverted to a different airfield for landing.

This was the second time in three days that the right-side engine had needed to be shut down. During a test flight after an engine change on 27 December, the pilot had noticed falling engine rpm, a climbing exhaust gas temperature gauge and a reduction in torque. All of these announced themselves as the aircraft was at 500 feet on final approach. Two days later a similar event happened in cruise.

To mangle a quote from Oscar Wilde: ‘To lose one may be regarded as misfortune; to lose both looks like carelessness’. But carelessness is a loaded word. It sounds too much like the pejorative and unilluminating ‘pilot error’. The background is likely to be a much more insidious condition described by Professor Sidney Dekker as ‘drift into failure’.

The lubrication system on theTPE331 engine uses three scavenge pumps and one pressure pump. One of these pumps oil from the rear turbine to the reduction gearbox, to ensure it gets lubricated. The other two scavenge pumps recover oil from the gearbox and pump it through an oil cooler into the oil tank, ready for use by the pressure pump. The oil flows in a circuit round the engine. For start-up there is an oil vent valve which lets air into the pumps. This lessens what would otherwise be a considerable load on the starter-generator during engine start. On this particular aircraft installation, the oil vent and oil cooler hoses are next to each other and similarly sized.

An investigation of the second incident found that the oil hoses had been incorrectly connected during the right-engine installation. The hose from the oil cooler to the oil tank had been fitted to the oil vent and the oil vent-oil tank hose had been fitted to the oil cooler. As a matter of routine the engineers looked at the left engine—it too had the hoses wrongly connected. The result was a gradual but inevitable oil loss as the engine ran.

To its credit, the operator carried out a thorough investigation. It described the root cause of the in-flight shutdown as a maintenance error. Many of the engineers involved had had decades of experience and there was no extreme pressure to have the aircraft completed and more than sufficient time allocated for the task.

Among the investigation’s findings were:

  • The aircraft maintenance engineer (AME) who fitted the hoses was not experienced on the particular aircraft and engine type.
  • The LAME did not double-check the hose position for correctness, since he knew it was installed ‘the same as the other engine’.
  • Some instructions for continuing airworthiness (ICA) that were relevant as a warning for this task had not been included in the work package.
  • The design of the oil hose installation for this particular airframe/engine configuration allowed for human factor errors. It was too easy for the unlearned to fit the hoses incorrectly—they were the same size, flexible hoses were not marked/colour coded etc.
  • The AME was unsure how to access ICAs/manuals from the company’s computerised maintenance database.
  • An engine manufacturer’s service information letter (SIL P331-115) had been circulated about the concern in 1990. However, it had not been included as a warning in current revisions to the Cessna 441 aircraft maintenance manual.

There were clues to this incident that seem significant in retrospect. The oil cooler had to be replaced twice before the 29 December flight, once after a ground run of the new engine, and once after the 27 December test flight. No direct connection was drawn between the two oil system problems and a potential common cause contributing to the failures.

Both maintenance teams were working on both engines, increasing the chance of an error being carried through both engines during maintenance. Independent LAME inspections for critical control systems would have been required for the engine changes/installation as a complete task but it was not picked-up that both engines could have their oil hoses incorrectly fitted in the same way. The maintenance organisation has now included in its system of maintenance both a warning and an independent (duplicate) inspection for this task of hose installation.

But hindsight is too easy, and assists mere judgement rather than true insight. ‘People who know the outcome of a complex history of tangled, indeterminate events, remember that history as being much more determinant, leading “inevitably” to the outcome they already knew,’ says Karl Wieck in his 1995 study: Sensemaking in Organisations.

‘We read a story of drift into the data, not out of the data,’ Dekker says.

Dekker’s concept of drift into failure boils down to a nuanced version of the proverb that pride comes before a fall.

‘In stories of drift into failure organisations fail precisely because they are doing well—on a narrow range of performance criteria, that is—the ones that get rewarded in the current political, economic or commercial configuration,’ he writes.

The defence against this is to beware of success. It inevitably involves some sort of unnoticed organisational change to cope with it, Dekker argues. The parts of an organisation that are booming are the ones that deserve scrutiny.

Again, Oscar Wilde has the witty last word. What else but system drift could he have been thinking of when he wrote: ‘we should treat all trivial things in life very seriously, and all serious things of life with a sincere and studied triviality’?

Share this article
Print this pageEmail this to someoneShare on Facebook0Tweet about this on TwitterShare on LinkedIn34

Leave a Reply