RCA Tool Comparison

As a discipline, Root Cause Analysis (RCA) has been approached from two different areas, industrial safety or performance improvement. The industrial safety viewpoint is oriented primarily at preventing bad things, while the performance improvement viewpoint is aimed at producing good things. There is overlap between the two priorities, but overall, the differing viewpoints have led to the development of different "schools" of RCA, with different tools and philosophies.

There has historically been extensive research and development dedicated to RCA tools for industrial safety (worker safety, process safety). The requirements are well-known, a wide variety of tools have been developed, and the strengths and weaknesses of specific approaches are understood. (This is not to say that the tools are perfect, because they're not.) However, the story is a little different in the performance improvement area. The theoretical underpinnings are generally not as well-developed, and while there are a number of tools available, there is less knowledge about the usefulness of the various tools.

A recent study by Dr. Anthony Mark Doggett [Ref 1] tries to improve the state of knowledge regarding three tools used widely in the performance improvement school of RCA: the cause-effect diagram (CED), the interrelationship diagram (ID), and the current reality tree (CRT). The purpose of the study was to "...compare the perceived differences... with regard to causality, factor relationships, usability, and participation." In doing so, Doggett attempts to address the perception that "...one tool is as good as another tool."

Note: Please have a look at my RCA Tools page if you're interested in detailed information on other tools.

Statistical Results

A key feature of this study is that it is qualitative, and measures perceived differences between the tools. The measurements were obtained by having several groups of college students actually perform RCAs. They were introduced to the tools, given opportunities to ask questions, and then presented with a problem and asked to "...find the perceived root cause of the problem." Afterwards, the students' perceptions were captured using question surveys and analyzed statistically.

Participation: No statistical differences (between the 3 tools) were perceived regarding the ability to spark constructive discussion in a group setting.
Causality: No statistical differences were perceived regarding the ability to identify interdependencies between causes, or to find root causes.
Factors: No statistical differences were perceived regarding the ability to find factors (causes, effects, or both), or relationships between them. However, post-hoc testing showed that the CED was perceived to be better at categorizing factors.
Usability: There were significant statistical differences observed in this area. Generally, the CRT was judged to be much harder to use than both the CED and the ID.

Root Cause Results

Beyond the statistical results, the study examined the ability of the students to identify root causes that were specific and reasonable. Note that this factor was examined separately from the usability factor discussed above.

CED: In general, students using the CED were not able to identify specific root causes, even though they perceived it to be better at "... facilitating productive problem-solving activity, being easier to use, and more readable."
ID: Students using the ID were able to find (i.e., identify and agree upon) root causes, but they were of mixed quality as regards specificity and reasonability. Otherwise, the ID was perceived to be no worse than the CED, in general.
CRT: The students perceived the CRT as complex and difficult to use. However, even though most students using the CRT were uncomfortable doing so, the quality of their outputs was better. They were able to find root causes most of the time, and with high integrity in over half the cases.

Conclusion

Regarding the findings for the CRT, Doggett states that "This was one of the distinguishing findings of the study." He stops short of saying this in his conclusions. Nevertheless, the finding is still immensely valuable -- even though the CRT is complex and more difficult to use, analysts employing it are more likely to find accurate root causes. Coupled with the team problem-solving dynamic, and the new thought processes introduced, that is the most important consideration.

References

A Statistical Comparison of Three Root Cause Analysis Tools - Dr. Anthony Mark Doggett, Journal of Industrial Technology, April 2004. [PDF]
CED: Cause and Effect Diagram
ID: Interrelationship Digraph
CRT: Current Reality Tree

Previous article: The BOGUS Test

Next article: The Fortunate Near-Miss Event

Comments on RCA Tool Comparison