mainlogo
Tool Catalogue for Uncertainty Assessment
Introduction
1. Sensitivity analysis
2. Error propagation equation ("TIER 1")
3. Monte Carlo analysis ("TIER 2")
4. NUSAP
Description
Goals and use
Sorts and locations of uncertainty addressed
Required resources
Strengths and limitations
Guidance on application
Pitfalls
References
5. Expert elicitation for uncertainty quantification
6. Scenario analysis
7. PRIMA: A framework for perspective-based uncertainty management
8. Checklist for model quality assistance
9. A method for critical review of assumptions in model-based assessments

Strengths and limitations

Typical strengths of NUSAP are:

  • NUSAP identifies the different sorts of uncertainty in quantitative information and enables them to be displayed in a standardized and self-explanatory way. Providers and users of quantitative information then have a clear and transparent assessment of its uncertainties.
  • NUSAP fosters an enhanced appreciation of the issue of quality in information. It thereby enables a more effective criticism of quantitative information by providers, clients, and also users of all sorts, expert and lay.
  • NUSAP provides a useful means to focus research efforts on the potentially most problematic parameters by identifying those parameters, which are critical for the quality of the information.
  • It is flexible in its use and can be used on different levels of comprehensiveness: from a 'back of the envelope' sketch based on self elicitation to a comprehensive and sophisticated procedure involving structured informed in-depth group discussions on a parameter by parameter format, covering each pedigree criterion combined with a full blown Monte Carlo assessment.
  • The diagnostic diagram provides a convenient way in which to view each of the key parameters in terms of two crucial attributes. One is their relative contribution to the sensitivity of the output, and the other is their strength. When viewed in combination on the diagram, they provide indications of which parameters are the most critical for the quality of the result.

Typical weaknesses of NUSAP are:

  • The method is relatively new, with a limited (but growing) number of practitioners. There is as yet no system of quality assurance in its applications, nor settled guidelines for good practice.
  • The scoring of pedigree criteria is to a certain degree subjective. Subjectivity can partly be remedied by the design of unambiguous pedigree matrices and by involving multiple experts in the scoring. The choice of experts to do the scoring is also a potential source of bias.
  • The method is applicable only to simple calculations with small numbers of parameters. But it may be questioned whether complicated calculations with many parameters are capable of effective uncertainty analysis by any means.