A new class of aviation accidents threatens to make experience count for nothing—unless we learn from it.
Professor Patrick Hudson knows the value of a good opening line. ‘When I first flew, in 1954, no one would have been that surprised if I hadn’t reached New York, my destination, from London, in a Constellation,’ the safety consultant to the oil and gas industry told this year’s Safeskies conference in Canberra. He conceded that ‘people would have been very upset, maybe, but not completely surprised.’
Hudson’s thesis is that the safety triumph of air transport in the 61 years since then, has led to a paradox: far fewer accidents, but a disturbing lack of pattern and predictability in those that still occur. They are, literally, weird.
‘The sort of disasters I am talking about may include Air France AF 447, Asiana 214, Turkish Airlines TK 1951 and Colgan CO3407, while the successful recoveries include the ‘miracle on the Hudson, QF 32, BA 38 and Air Transat 236. We do still have accidents of the ‘old’ sort, but the weird ones now stand out and we have found that WEIRD can be an acronym,’ he told the Safeskies delegates.
Weird stands for:
R resulting in
Competence, experience and survival
Because not all weird events cause crashes, Hudson asks, ‘What made the difference, in these sorts of incidents, between disaster and relief? He is not convinced by the usual answer—experience.
‘When I read the reports, but more often the commentaries by experts, I see references to the word experience. Either the pilots were technically competent, but respectively lacked or possessed the experience to avoid danger or manage what was almost always the unexpected consequence of some failure. Yet all the pilots had enough hours to be considered sufficiently experienced.’
Hudson has two problems with experience as a panacea. Firstly, he wonders whether modern, reliable, aircraft generate it in the way their cantankerous forebears did.
‘While the number of hours logged used to be a good proxy for experience, today that link has become almost completely broken,’ he says. ‘In Canadian offshore helicopter operations, I have been told that pilots flew an average of 450 hours a year, of which 14 were hands on. I don’t expect long-range commercial flying to be much better. ‘
Secondly, he argues that experience is different to competence, a relatively simple concept that can be defined, examined and generated with formal systems. Experience, in contrast, is the legacy of incidents when a pilot pushed the limits of their competence, or their luck. ‘We might crudely define experience as the number of exposures to potential disaster that an individual had successfully survived,’ he says, leaving unsaid the obvious point that this is a dangerous way to learn.
‘What we have come to expect is that training delivers competence that can be equated to experience, but maybe it’s not as simple as that. Likewise, experience may not always get us competence. The two can and should be distinguished, and conflating them may have consequences that we would rather avoid.’
Accident causation models
Hudson proposes that there are three types of accident. Simple ones, with readily definable causes, he labels Type I. ‘We can use the word “cause” without losing sleep over it at this level,’ he says. ‘More subtle accidents, with more subtle causes are Type II. The cause-effect relationship is less clear in these and models such as Reason’s Swiss cheese help explain the contributing factors. Type III accidents have non-deterministic causal factors. ‘Here we can talk at best about factors that impact on the results, avoiding the cause word altogether.’
- Type I
Simple models may cover 80 per cent of all accidents
- Type II
The next step gets 80 per cent of the remainder = 16 per cent.
Type I +Type II = 96 per cent of accidents
- Type III
The probabilistic approach may net the next 80 per cent = 3.2 per cent. Types I+II+III= 99.2 per cent of accidents.
Hudson adds a caution: with hindsight everything looks like Type I
Using simple mathematics, Hudson demonstrates that while Type III incidents are a small proportion of recorded accidents, they are the overwhelming majority of potential accident scenarios. The three-dimensional graph representing recorded and potential accidents is like a Mexican sombrero, with the crown of the hat representing recorded accidents, and the much wider brim representing thousands of potential accident scenarios.
Aviation has been remarkably successful at reducing Type I and Type II accidents but society has become intolerant of any air transport accident, Hudson says. ‘Society has gone and moved the goalposts: what people once would have shrugged over has now become completely unacceptable.’
Same old, same old …
Discipline, drills and the checklist proved effective against Type 1 accidents in the 1950s, and in the 1980s, crew resource management, (CRM), became an established defence against Type II accidents. But Hudson is sceptical that CRM, particularly as currently taught, can protect against Type III accidents. The problem is partly in how it has come to be taught.
‘We have expected current approaches to CRM to generalise to Type III. This may not always be the case. Modern CRM training, I am afraid, has all too often become a drill exercise, reduced to the same methodology as technical drills. The first limitation I can identify with today’s approach to CRM is that it is typically passive—talk and chalk multiplied by death by PowerPoint.’
We are trying to prevent the ‘new’ accidents using more of the same ‘old’ tools.
Fighting weird with weirdness: training for Type III
The whole point about weird scenarios is that they are unusual and unexpected. ‘The chances of coming up against an exact scenario is vanishingly small, so what you have to learn are generalisable skills,’ Hudson says.
Decision-making is one such skill. ‘Decision-making is difficult and there is a considerable scientific literature about it. There is a literature about how people make good decisions under non-ideal circumstances; we need to make more use of that.’
Hudson sees potential in studies of naturalistic decision making, the recognition-primed decision model of Gary Klein and the ‘fast and slow thinking,’ ideas of Danny Kahneman and Daniel Tversky.
Medicine, which normally lags aviation in human factors has one shining light, Hudson says. ‘ATLS stands for Advanced Trauma Life Saving and is designed for just the kinds of situation aviation isn’t doing to well at.’
- Physical practice of scenarios.
- Assessment and scoring, often using video analysis.
- Realism, often using moulage, a French word meaning mock wounds.
Sport is an area where the experience versus attrition debate is acute. Rugby, at the level of the World Cup, cannot be learned from a book, but excessive match practice can quickly injure players or weary entire teams.
Visualisation is the technique top sports players use to solve this problem. ‘You run a play, a sequence of corners, part of a match or a race even in your mind. A conscious disciplined exploration of a scenario, one you can run and rerun without getting hurt, or damaging your racing car in Formula 1.’ says Hudson.
‘Done properly this is no mere daydreaming, but a mental activity that, surprisingly, can be shown to have effects equivalent to having the physical experience. This is something in common with the trauma training with physical involvement, moulage and time stress.’
Use your imagination—for safety’s sake
‘So, how can we generate enough scenarios to be useful, because what we need is bulk and variety? Hudson asks. ‘We need imagination, a commodity hard to regulate for, but I am sure most pilots have some.
‘Two locations for this new type of training come immediately to mind, but there will be more. The first is on the flight deck, whiling away the time by indulging in some active imagination of how things could go wrong, trying to come up with weird but feasible combinations of factors that a malign world might throw at you. The best can and should be recorded somehow: tablets are good for that. People could share their wildest dreams and vote on the best, all ways of spreading the knowledge around the wider community.
‘The second is a location where many pilots find themselves, and where they already show this kind of behaviour: the bar after a long day’s flying. Wherever two or more pilots are joined together they will talk about flying; let’s make use of that.
‘Anyone who says, “that’s what we do already” is already agreeing with me. Inventing scenarios, then visualising what to do about them, in my book that probably makes you more experienced than your less imaginative colleagues, regardless of their hours. All we need to do is to bring everyone else up to speed.’