Evaluating the Quality of a Root Cause Analysis
The quality of a Root Cause Analysis (RCA) and its Corrective Action Plan (CAP) should be evaluated many times over its lifecycle (i.e., from initial problem or event, through to final verified and sustainable improved state). Reviews occurring earlier in the lifecycle can really consider only the apparent quality of the investigation/analysis effort itself; these early reviews are what I will discuss in this article.
So, here's what I think that early RCA & CAP reviews should check.
- Is the problem or event clearly and concisely defined in a manner that captures the major consequences, current and potential (on-going) risk factors, and costs to the organization (e.g., money, time, reputation, goodwill, competitive advantage)?
- Is there sufficient depth and breadth of factual evidence to establish what happened, how it happened, and why it happened?
- Has the investigation gone deep enough to find the causes of the problem or event while ensuring that the selected causes or causal factors are backed up by adequate evidence and rigorous application of Necessary and Sufficient logic tests?
- Is the logic supporting the determination of causes consistent, clearly defined, tightly linked, and well developed with no evidence of logical fallacies (e.g., single chain of causation) or cognitive biases (e.g., hindsight)?
- Were extent of condition and extent of cause considered to the level necessary to ensure that any generic/related risks have been identified and dealt with appropriately?
- Are the root causes, contributing causes, organizational weakness, other causal factors, and lessons-learned clearly and concisely described in a manner that flows logically from the conclusions of the analysis, and do they make sense if viewed in reverse (i.e., start from cause statements and work back to the original problem or event)?
- Was available operating experience (internal or external) used constructively to support the analysis, conclusions, and corrective action recommendations?
- Did the investigation team consider and address any organizational processes that could have caught or prevented the problem or event before it happened (e.g., from past investigations, audits, assessments)?
- Do the corrective actions naturally and clearly address the issues brought out by the causes and lessons-learned?
- Are the primary corrective actions of sufficient gravity to prevent recurrence, or merely token actions aimed at alleviating the symptoms of the problem?
- Are the remaining corrective actions sufficient to ensure that the possibility of recurrence is minimized until the primary actions are completed?
- Are the corrective actions specific, measurable, achievable, relevant, and timely (SMART) and are they all assigned to the right people/organizations?
- Do the corrective actions define success criteria or provide a picture of what "good" looks like?
- If the entire corrective action plan is performed as written, and as scheduled, does it appear that the organization will have implemented the right changes to result in lasting improvement?
Who should do this review? I believe it should be done by several people at several different times during the investigation. The investigation team should do it, and at least one or two completely independent people with root cause and subject matter expertise should too. Then, depending on the significance of the original problem or event, the sponsoring manager should review it a couple of times before passing it on to a small group of senior managers for review and discussion. Whenever any deficiencies or unacceptable residual risks are identified, they should be addressed and the revised product reviewed again.
Does that seem like too much, or not enough? Or is it just right? Let me know in the comments!
by Bill Wilson