When risk estimates are expressed as single values–rather than as probability distributions–the level of conservatism used in producing those estimates must be declared. This is an essential element in risk assessment.
Conservatism is generally taken to mean an intentional bias towards over-estimation of the true risk. Risk assessment incorporating a high level of conservatism will tend to overstate the risks, perhaps by several orders of magnitude. This occurs through the use of input values and calculations that are based on worst-case or at least ‘higher risk’ assumptions. Risk assessment conducted with no conservatism always assumes the most likely values and the results are most often true to the most common actual conditions. So, uncommon conditions are missed unless conservatism is used.
As used here, conservatism includes addressing uncertainty. All measurements, including estimates of risk, should address uncertainties due to measurement inaccuracies.
Conservatism is a useful characteristic in many applications of risk management. However, conservatism may also be excessive, leading to inefficient and costly choices when not properly acknowledged in decision-making.
A risk assessment producing discreet estimates of risk should be performed with a target level of conservatism. As used here, the PXX designations indicate a level of confidence that actual experience will be no worse than estimated. For instance, P90 is the point where 90% of future performance should be at or below this value. It is the point where one would be negatively surprised 10% of the time—once out of every ten episodes.
A P90+ assessment intentionally contains layers of conservatism. This is often done to encourage future data collection as a means of risk reduction and, more importantly, to ensure that risks are not underestimated. Underestimation of risk prevents sound risk management and results in loss of credibility.
For simplicity, the PXX refers to the conservatism of inputs rather than to the resulting conservatism of the assessment. Each risk assessment is obtained via a collection of inputs, each with an estimated level of uncertainty equal to PXX. Actual conservatism of final risk estimates often increase dramatically due to layering of conservatism due to a bias towards conservatism for each input. This layering also produces increasingly unlikely scenarios since multiple low probability events are assumed to occur simultaneously. Therefore, the PXX refers to the intended level of uncertainty associated with each input rather than the risk estimates. The PXX level of the final risk assessment is identified and managed in the calibration/validation phase (see ).
Less conservative assumptions are sometimes needed for practical reasons. For instance, a defect over 95% through a pipe wall could exist and survive a pressure test or be undetected in an inspection. It would be counter-productive to assume that such rare defects exist everywhere, even though such an assumption would be very conservative. Rather, the wall thickness implied by a Barlow stress calculation (perhaps adjusted by a factor showing some localized thinning could have occurred) can be used as the primary means to estimate the probable—and still conservative—wall thickness when no other confirmatory integrity information is available.
P84, as representing approximately one standard deviation of a one-sided normal distribution, might be appropriate as a target level of conservatism that may have some consistency with certain design practices.
Some practitioners also produce P10 or similar estimates, reflecting best case or at least more optimistic inputs. As with the more conservative layering, choosing multiple optimistic inputs produces a combination with even more optimism, as well as more rarity.
The user should determine the level of conservatism appropriate to his needs. Often a P99 level—negative surprises only 1% of the time—or higher is warranted for assessments supporting new projects or presentations in public forums. A P50 to P70 level of analysis might be more appropriate for budget setting or long range planning.
It is recommended that conservative, perhaps P90+, risk assessment results be used for all location-specific risk management. Less conservative, such as P50, results are useful for strategic planning and communicating with certain stakeholders.
Article: Controlling the Bias
See also Uncertainty and the discussion of calibration in Chapter 3.