Suchergebnisse

This book is an expository introduction to the methodology of sensitivity analysis of model output. It is primarily intended for investigators, students and researchers that are familiar with mathematical models but are less familiar with the techniques for performing their sensitivity analysis. A variety of sensitivity methods have been developed over the years. This monograph helps the analyst in her/his first exploration of this world. The main goal is to foster the recognition of the crucial role of sensitivity analysis methods as the techniques that allow us to gain insights from quantitative models. Also, exercising rigor in performing sensitivity analysis becomes increasingly relevant both to decision makers and modelers. The book helps the analyst in structuring her/his sensitivity analysis quest properly, so as to obtain the correct answer to the corresponding managerial question. The first part of the book covers Deterministic Methods, including Tornado Diagrams; One-Way Sensitivity Analysis; Differentiation-Based Methods and Local Sensitivity Analysis with Constraints. The second part looks at Probabilistic Methods, including Regression-Based methods, Variance-Based Methods, and Distribution-Based methods. The final section looks at Applications, including capital budgeting, sensitivity analysis in climate change modelling and in the risk assessment of a lunar space mission.

In this work, we study the effect of epistemic uncertainty in the ranking and categorization of elements of probabilistic safety assessment (PSA) models. We show that, while in a deterministic setting a PSA element belongs to a given category univocally, in the presence of epistemic uncertainty, a PSA element belongs to a given category only with a certain probability. We propose an approach to estimate these probabilities, showing that their knowledge allows to appreciate "the sensitivity of component categorizations to uncertainties in the parameter values" (U.S. NRC Regulatory Guide 1.174). We investigate the meaning and utilization of an assignment method based on the expected value of importance measures. We discuss the problem of evaluating changes in quality assurance, maintenance activities prioritization, etc. in the presence of epistemic uncertainty. We show that the inclusion of epistemic uncertainly in the evaluation makes it necessary to evaluate changes through their effect on PSA model parameters. We propose a categorization of parameters based on the Fussell‐Vesely and differential importance (DIM) measures. In addition, issues in the calculation of the expected value of the joint importance measure are present when evaluating changes affecting groups of components. We illustrate that the problem can be solved using DIM. A numerical application to a case study concludes the work.

In this work, we introduce a generalized rationale for local sensitivity analysis (SA) methods that allows to solve the problems connected with input constraints. Several models in use in the risk analysis field are characterized by the presence of deterministic relationships among the input parameters. However, SA issues related to the presence of constraints have been mainly dealt with in a heuristic fashion. We start with a systematic analysis of the effects of constraints. The findings can be summarized in the following three effects. (i) Constraints make it impossible to vary one parameter while keeping all others fixed. (ii) The model output becomes insensitive to a parameter if a constraint is solved for that parameter. (iii) Sensitivity analysis results depend on which parameter is selected as dependent. The explanation of these effects is found by proposing a result that leads to a natural extension of the local SA rationale introduced in Helton (1993). We then extend the definitions of the Birnbaum, criticality, and the differential importance measures to the constrained case. In addition, a procedure is introduced that allows to obtain constrained sensitivity results at the same cost as in the absence of constraints. The application to a nonbinary event tree concludes the article, providing a numerical illustration of the above findings.

AbstractQuantitative models support investigators in several risk analysis applications. The calculation of sensitivity measures is an integral part of this analysis. However, it becomes a computationally challenging task, especially when the number of model inputs is large and the model output is spread over orders of magnitude. We introduce and test a new method for the estimation of global sensitivity measures. The new method relies on the intuition of exploiting the empirical cumulative distribution function of the simulator output. This choice allows the estimators of global sensitivity measures to be based on numbers between 0 and 1, thus fighting the curse of sparsity. For density‐based sensitivity measures, we devise an approach based on moving averages that bypasses kernel‐density estimation. We compare the new method to approaches for calculating popular risk analysis global sensitivity measures as well as to approaches for computing dependence measures gathering increasing interest in the machine learning and statistics literature (the Hilbert–Schmidt independence criterion and distance covariance). The comparison involves also the number of operations needed to obtain the estimates, an aspect often neglected in global sensitivity studies. We let the estimators undergo several tests, first with the wing‐weight test case, then with a computationally challenging code with up to inputs, and finally with the traditional Level E benchmark code.

AbstractRisk‐informed decision making is often accompanied by the specification of an acceptable level of risk. Such target level is compared against the value of a risk metric, usually computed through a probabilistic safety assessment model, to decide about the acceptability of a given design, the launch of a space mission, etc. Importance measures complement the decision process with information about the risk/safety significance of events. However, importance measures do not tell us whether the occurrence of an event can change the overarching decision. By linking value of information and importance measures for probabilistic risk assessment models, this work obtains a value‐of‐information‐based importance measure that brings together the risk metric, risk importance measures, and the risk threshold in one expression. The new importance measure does not impose additional computational burden because it can be calculated from our knowledge of the risk achievement and risk reduction worth, and complements the insights delivered by these importance measures. Several properties are discussed, including the joint decision worth of basic event groups. The application to the large loss of coolant accident sequence of the Advanced Test Reactor helps us in illustrating the risk analysis insights.

Information value has been proposed and used as a probabilistic sensitivity measure, the idea being that uncertain parameters having higher information value are precisely those to which an optimal decision is more sensitive. In this paper, we study the notion of information density as a graphical complement to information value analysis, one that augments an information value calculation with associated directions of information gain. We formally examine mathematical details absent from its earlier presentation that guarantee information density exists and is well posed and describe its relationship to alternate measures of information value. We present its application in the context of a realistic case study and discuss the associated insights.

Moment independent methods for the sensitivity analysis of model output are attracting growing attention among both academics and practitioners. However, the lack of benchmarks against which to compare numerical strategies forces one to rely on ad hoc experiments in estimating the sensitivity measures. This article introduces a methodology that allows one to obtain moment independent sensitivity measures analytically. We illustrate the procedure by implementing four test cases with different model structures and model input distributions. Numerical experiments are performed at increasing sample size to check convergence of the sensitivity estimates to the analytical values.

AbstractIn probabilistic risk assessment, attention is often focused on the expected value of a risk metric. The sensitivity of this expectation to changes in the parameters of the distribution characterizing uncertainty in the inputs becomes of interest. Approaches based on differentiation encounter limitations when (i) distributional parameters are expressed in different units or (ii) the analyst wishes to transfer sensitivity insights from individual parameters to parameter groups, when alternating between different levels of a probabilistic safety assessment model. Moreover, the analyst may also wish to examine the effect of assuming independence among inputs. This work proposes an approach based on the differential importance measure, which solves these issues. Estimation aspects are discussed in detail, in particular the problem of obtaining all sensitivity measures from a single Monte Carlo sample, thus avoiding potentially costly model runs. The approach is illustrated through an analytical example, highlighting how it can be used to assess the impact of removing the independence assumption. An application to the probabilistic risk assessment model of the Advanced Test Reactor large loss of coolant accident sequence concludes the work.

AbstractRisk analysts are often concerned with identifying key safety drivers, that is, the systems, structures, and components (SSCs) that matter the most to safety. SSCs importance is assessed both in the design phase (i.e., before a system is built) and in the implementation phase (i.e., when the system has been built) using the same importance measures. However, in a design phase, it would be necessary to appreciate whether the failure/success of a given SSC can cause the overall decision to change from accept to reject (decision significance). This work addresses the search for the conditions under which SSCs that are safety significant are also decision significant. To address this issue, the work proposes the notion of a θ‐importance measure. We study in detail the relationships among risk importance measures to determine which properties guarantee that the ranking of SSCs does not change before and after the decision is made. An application to a probabilistic safety assessment model developed at NASA illustrates the risk management implications of our work.

Measures of sensitivity and uncertainty have become an integral part of risk analysis. Many such measures have a conditional probabilistic structure, for which a straightforward Monte Carlo estimation procedure has a double‐loop form. Recently, a more efficient single‐loop procedure has been introduced, and consistency of this procedure has been demonstrated separately for particular measures, such as those based on variance, density, and information value. In this work, we give a unified proof of single‐loop consistency that applies to any measure satisfying a common rationale. This proof is not only more general but invokes less restrictive assumptions than heretofore in the literature, allowing for the presence of correlations among model inputs and of categorical variables. We examine numerical convergence of such an estimator under a variety of sensitivity measures. We also examine its application to a published medical case study.