mainlogo
Tool Catalogue for Uncertainty Assessment
Introduction
1. Sensitivity analysis
2. Error propagation equation ("TIER 1")
3. Monte Carlo analysis ("TIER 2")
4. NUSAP
5. Expert elicitation for uncertainty quantification
Description
Goals and use
Sorts and locations of uncertainty addressed
Required resources
Strengths and limitations
Guidance on application
Pitfalls
References
6. Scenario analysis
7. PRIMA: A framework for perspective-based uncertainty management
8. Checklist for model quality assistance
9. A method for critical review of assumptions in model-based assessments

Strengths and limitations

Strengths

  • It has the potential to make use of all available knowledge including knowledge that cannot be easily formalized otherwise.
  • It can easily include views of sceptics and reveals the level of expert disagreement on certain estimates.

Weaknesses

  • The fraction of experts holding a given view is not proportional to the probability of that view being correct.
  • One may safely average estimates of model parameters, but if the expert's models were incommensurate, one may not average models (Keith, 1996).
  • If differences in expert opinion are irresolvable, weighing and combining the individual estimates of distributions is only valid if weighted with competence of the experts regarding making the estimate. There is no good way to measure competence. In practice, the opinions are often weighted equally, although sometimes self-rating is used to obtain a weight-factor for the experts competence
  • The results are sensitive to the selection of the experts whose estimates are gathered.

Although subjective probability is an imperfect substitute for established knowledge and despite the problems of aggregation of expert judgement, if nothing better is available it is better to use subjective probability distributions than deterministic point-values so that one has at least a first approximation of the uncertainty.