mainlogo
Tool Catalogue for Uncertainty Assessment
Introduction
1. Sensitivity analysis
2. Error propagation equation ("TIER 1")
3. Monte Carlo analysis ("TIER 2")
4. NUSAP
Description
Goals and use
Sorts and locations of uncertainty addressed
Required resources
Strengths and limitations
Guidance on application
Pitfalls
References
5. Expert elicitation for uncertainty quantification
6. Scenario analysis
7. PRIMA: A framework for perspective-based uncertainty management
8. Checklist for model quality assistance
9. A method for critical review of assumptions in model-based assessments

Pitfalls

Typical pitfalls of NUSAP are:

  • Misinterpreting low pedigree scores as indicating low-quality science. In relation to whole disciplines, this amounts to 'physics-envy'. Quality in science depends not on removing uncertainty but on managing it.
  • Misinterpreting pedigree scores as an evaluation of individual items of information, with low scores indicating bad research. The pedigree analysis is of the characteristic limits of knowledge of areas of inquiry. The quality of individual items of information depends crucially on the craftsmanship of the work, requiring a closer analysis, which the pedigree does not undertake.
  • Motivational bias towards high pedigrees in (self) elicitation, especially in case of numbers where one or one's institute was involved in the knowledge production. This pitfall is avoided by the use of trained facilitators in an open process for the construction and assignment of pedigrees.
  • Falsely thinking that pedigree and spread are correlated: In principle these are independent dimensions.