Richard Metcalfe, Quintessa Limited (United Kingdom)
Philip Maul, Quintessa Limited (United Kingdom)
Steven Benbow, Quintessa Limited (United Kingdom)
Claire Watson, Quintessa Limited (United Kingdom)
David Hodgkinson, Quintessa Limited (United Kingdom)
Alan Paulley, Quintessa Limited (United Kingdom)
Laura Limer, Quintessa Limited (United Kingdom)
Russell Walke, Quintessa Limited (United Kingdom)
David Savage, Quintessa Limited (United Kingdom)
Most generally, Performance Assessment (PA) evaluates the performance of a specified system or sub-system relative to some criterion of interest to a particular stakeholder. The performance measure(s) depend upon the stakeholders and their goals. A regulator interested in safety may assess an entire storage system and its surroundings, and the performance indicator may be a health risk. Alternatively, a reservoir engineer might consider only the reservoir and cap rock, and performance measures might be injectivity, storage capacity and caprock sealing efficiency. Stakeholders interested in operational safety will usually consider timescales of tens of years; those evaluating whether geological storage will mitigate climate change may need to evaluate periods in the order of 103 years.
Whatever a PA's scope, varied quantitative and qualitative information and multiple lines of reasoning must be used to build confidence in the results. The different kinds of information will to some degree have uncertain and/or conflicting implications for system performance. To enable a PA to be defended robustly, all information used and its application to develop safety arguments must be recorded.
For these reasons, a flexible framework has been developed for the structured integration of varied PA-relevant information. Decision trees are constructed to reflect: (1) the PA's context; (2) the Features, Events and Processes (FEPs) that may influence the evaluated system; and (3) the kinds of information used to judge the characteristics and effects of interactions among FEPs. The decision tree is a hierarchy of hypotheses, which links the main hypothesis of interest (e.g. insignificant CO2 will leak to the surface from a storage reservoir) to data or information (e.g. geological observations, experimental measurements, simulation output etc), usually via intermediate hypotheses. Inputs are independent numerical representations of evidence for and against the dependability of the hypotheses at the hierarchy's lowest level. The evidence may correspond to quantitative or qualitative information and is propagated through the tree using Evidence Support Logic (ESL), which is based on Interval Probability Theory (IPT). Using IPT allows uncertainties to be represented in addition to the evidence for and against a hypothesis being correct, which is all that can be treated by classical analyses. Uncertainties can be analysed to identify those that impact upon an overall decision most significantly, enabling future information acquisition can be prioritized.
The framework has been implemented in two inter-linked software tools: (1) TESLA, which provides an environment for constructing decision trees, propagating evidence and recording the logic underlying a decision; and (2) an On-line FEP Database. The knowledge base represented by the FEP database may be accessed directly from a decision tree.