Accident statistics: valid?

Statistical analyses of accidents has troubled me for many years. One of my main objections was that statistical correlation does not equate to "cause." Very recently an even more significant insight occurred to me during a discussion with a colleague, Ira Rimson, as we were discussing the system descriptions as inputs used by safety analysts. We were discussing the description requirements to help safety analysts understand the dynamics of the systems they were being asked to analyze, and how those dynamics should be described. Our experience suggested that the descriptions presently offered were fragmented elements at best. Extending that notion to the descriptions of accidents led to a new concern involving the idea of "sampling" accident dynamics during accident investigations. What should be sampled and how should the samples be documented?

Digitization of music provides an instructive analogy.

To digitally reproduce music, states of a musical work are sampled at various rates, typically ranging from 22k to 44k or more samples per second. The lower the sampling rate, the less faithfully the music is reproduced. Carried to its logical conclusion, a single sample of a song is useless if one wants to "hear" the data as music.

Think of the production of music or a song as a process, requiring the dynamic interactions of the people, and instruments, and the pitch, frequency and other constantly changing relationships necessary to produce the notes as the song progresses from beginning to end. To digitize the music the state of each of these attributes must be sampled frequently as the song progress. Digital video captures images sampling action in frames per second. Same idea: the greater the number of samples, the greater the fidelity of the reproduction.

Apply similar thinking to the descriptions of systems and their operation, and how such information is presented to system safety analysts who are expected to find hazards in a system and predict system aberrations. How well is it possible to meet those expectations if the descriptions of the processes they are to analyze has insufficient samples of the "music" the system operation might produce?

Now, think also of accidents as processes, involving dynamic interactions of the people, objects and energies needed to produce the accident as it progresses from its beginning to its outcome over time. What data do we sample to predict and capture a description of that process? That question has been answered after the fact in part in some activities, such as aircraft system operations captured with digital flight data recorders and analog cockpit voice recorders. But before the accidents happen, how can you predict them with insufficiently sampled process description?

The question seems worthy of more exploration.

Lessons Learning Systems

Accident investigation practices that produce "lessons learned" are coming under increasing scrutiny. Issues surfacing as a result of this scrutiny include how to streamline and maximize lessons learned development, documentation, dissemination, accessibility, internalization and feedback. Serious challenges to many aspects of current lessons learned systems are emerging from this scrutiny, including aspects such as the

• cause-based framework of present lessons learnedsystems
  
• orientation of accident investigations relative to lessons learned
  
• focus of lessons learned efforts,
• derivation of lessons learned during accident investigations
• maximization of the number of lessons learned
  
• language and structure of lessons learned documentation,
  
• latency in lessons learned cycles,
  
• context available with lessons learned,
  
• harmonizing of lessons learned data derived from accidents and other mishap investigation with lessons derived from other sources in an organization,
• data density of lessons learned outputs,
• breadth of the accessibility of lessons learned,
  
• internalization of lessons learned when accessed,
  
• monitoring of changes in activities attributable to lessons learned in mishaps,
  
• lessons learned life span,
• lessons learned obsolescence, and
• strategies for improving lessons learning systems performance

See http://www.iprr.org/research/llset/SSTS08_tutorial.pdf for a presentation about this topic. And chip in your two cent's worth if you have some useful thoughts.

Drifting toward failure

This expression is being recognized with growing frequency in major investigations like the Challenger loss. It reflects ideas about the responsibility of managers to direct their operations in a "safe" way, coming our of the human factors community.  A recent book called 10 Questions about Human Error, by Sidney Dekker, discusses it at length.

This  ideas behind the expression create some interesting challenges for mishap investigators. To name a few, what data should an investigator seek to illustrate this "drift" in a specific investigation? How does an investigator identify and acquire data that has to be coupled to the outcome of a mishap to show this "drift" in the context of when the mishap occurred? How should an investigator present the data to demonstrate its influence on the outcome in an objective, logical, persuasive and verifiable way? What risks are inherent in the investigating and reporting of this "drift" and who can be at risk of harm if the investigator does not do it properly? How can the quality of the reported drift be assessed or ensured? Are the answers primarily dependent on the investigator's experience or the methodology selected, or something else? 

As one ponders this expression and the ideas behind it, it becomes evident that they have application to many  kinds of activities other than mishaps. 

Kickoff blog 1

Here's a challenge.

I think anyone connected with safety or mishap investigations needs to rethink why we do investigations, rather than blindly accepting the folk wisdom about investigation purposes. I suggest that purposes like "to prevent recurrence!" or "find the cause or  probable cause or  root causes" or affix blame or find out what happened and why or find lessons learned or settle claims miss the ultimate purpose, and misdirect investigation practices.

I'd offer this simple candidate: TO CHANGE BEHAVIORS. The reason for investigating an occurrence is to find, understand and report the behaviors that produced an unwanted occurrence, and their context,  in a way that others engaged in similar behaviors can learn about them and apply that information to their future activities. 

I would argue that this purpose would change how investigators do investigations and what they produce, by focusing their inquiries on what people, objects or energies did during mishaps, and why they did that. I would argue further that this would improve future investigation efficiency, efficacy and value. I would also argue that this would produce better predictive understanding of mishap phenomena if it were applied to present system safety analysis practices.

Anyone want to defend the status quo?