| Project | Purpose/Scope | Update | % Complete | Stage | Target Date for Publication | Record # (TR)Ballot # (B) | VVUQ Standards Committee | VVUQ 1 – Verification, Validation, and Uncertainty Quantification Terminology in Computational Modeling and Simulation | ASME VVUQ 1 provides a harmonized set of definitions for verification, validation, and uncertainty quantification (VVUQ) concepts. | ANSI Approved: August 8, 2022 | 100% | Published | First Edition2022 | TR 19-2694B 21-617RC101 | VVUQ 10 | Verification, Validation, and Uncertainty Quantification in Computational Solid Mechanics | V&V 10 | Standard for Verification and Validation in Computational Solid Mechanics | The purpose of this Standard is to provide the CSM community with a common language, a conceptual framework, and general guidance for implementing the processes of computational model V&V. To this end, this Standard includes a glossary of terms, figures illustrating the recommended overall approach to V&V activities, and discussions of factors that should be considered when developing and executing a V&V program. To maximize the value to the engineering community, the ASME V&V 10 Subcommittee chose to write this Standard from the perspective of V&V for high-consequence computational predictions of complex engineering systems. However, the guidance provided here is also appropriate for simple applications, though it is understood that smaller budgets and lower risks will reduce the scope of the V&V effort. While the concepts and terminology presented here are applicable to all applied mechanics, the focus is on CSM. | ANSI Approved: July 23, 2019 | Published | Next Edition 2024 | VVUQ 10.1 | An Illustration of the Concepts of Verification and Validation in Computational Solid Mechanics (previous edition: V&V 10.1-2012) | The purpose of this document is to illustrate, by detailed example, the most important aspects of V&V described in the Committee’s framework document, Standard for Verification and Validation in Computational Solid Mechanics (V&V 10). | Before the revised draft can be balloted V&V 10.1-2012 needs to be reaffirmed since it was last approved by ANSI 10 years ago.The reaffirmation of V&V 10-2012 was ANSI approved on February 28, 2022. The standards committee/subcommittee ballot was disapproved and comments were submitted. | 50% | Revision Development | Next Edition 2022 | TR 22-713B 22-920 | VVUQ 10.2 | The Role of Uncertainty Quantification in Verification and Validation of Computational Solid Mechanics Models | This Standard describes the role of UQ in modeling/simulation and experimentation. UQ in modeling and simulation includes consideration of model-form uncertainties, numerical solution uncertainties, model input uncertainties, and uncertainties in model-basis data. In addition, propagation of uncertainties is an integral part of UQ in modeling and simulation. UQ plays an important role in experimentation, therefore key considerations in planning validation experiments are discussed, since these experiments are specifically planned and performed to assess the predictive capability of a computational model. A brief discussion of UQ in hierarchical CSM models is provided, as well as an overview of the role of UQ in revisions to either the computational model or the validation experiment. | ANSI Approved: December 13, 2021Published May 30, 2022 | 100% | ANSI Approved | First Edition 2021 | TR 19-1476B 19-1557RC101 | VVUQ 10.3 | The Role of Validation Metrics in Computational Solid Mechanics Models | The purpose of the present supplement is to provide a primer on quantitative metrics used within the validation process described in ASME V&V 10-2019. As a primer, this document aims at provides background, guidance and illustrative examples rather than an exhaustive compendium of possible metrics. The user is encouraged to treat this document as a pedagogic tool introducing the main metrics for practical use and associated limitations, as well as a guide for thoughtful selection and use of the most relevant metrics. | Meeting Monthly to discuss draft. | 40% | Draft Development | First Edition 2023 | TR-08-1328 | VVUQ 10.X | The Role of Verification, Validation, and Uncertainty Quantification in Credibility for the Decision-Making Process | Project Identified | V&V 10.4 - Role of Verification Methods in Verification and Validation of Computational Solid Mechanics Models | Project Identified | TR-09-440 | V&V 10.5 - Role of Validation Methods in Verification and Validation of Computational Solid Mechanics Models | Project Identified | TR-09-441 | V&V 10.6 - Defining Model Credibility for Intended Model Usage in Verification and Validation of ComputationalSolid Mechanics Models | Project Identified | TR-09-442 | V&V 10.7 - Role of Model Revision in Verification and Validation of Computational Solid Mechanics Models | Project Identified | TR-09-443 | V&V 10.8 - A End-to-End Example of Hierarchical Verification and Validation of Computational Solid Mechanics | Project Identified | TR-09-1435 | VVUQ 20 | Verification, Validation, and Uncertainty Quantification in Computational Fluid Dynamics and Heat Transfer | V&V20-2009 (Reaffirmed 2021) | Standard for Verification and Validationin Computational FluidDynamics and Heat Transfer | This standard provides a procedure to estimate the modeling error of a quantity of interest determined by a mathematical model used to simulate the same physical reality. The procedure can be applied to any quantity defined by a scalar and it produces an interval centered at the difference between the simulation and the experimental data that quantifies the physical reality with a width that depends on the experimental, numerical and input uncertainties. | There were two related papers developed on this topic:“On the Interpretation and Scope of the V&V 20 Standard for Verification and Validation in Computational Fluid Dynamics and Heat Transfer” (JVVUQ, March 24, 2022, paper VVUQ-20-1051)“Comparison of the V&V10.1 and V&V20 Validation Procedures for the V&V10.1 Example” (JVVUQ, March 15, 2022, paper VVUQ-22-1003) | Project defined | 2023/2024 | VVUQ 20.1 | Multivariate Metrics - Supplement 2 of ASME V&V 20 - Standard for Verification and Validation in Computational Fluid Dynamics and Heat Transfer | V&V 20-2009 presents a validation approach for estimating a range within which model error lies considering the uncertainties in the experiment, in the numerical solution of the model implementation, and in the simulation inputs. The committee which developed that document limited its initial consideration to validation of a specific variable at a single validation set point. The specific variable can be a directly-measured single variable, a dimensional variable determined from a combination of other measured variables, or a dimensionless variable (such as Nusselt number or friction coefficient) determined from a combination of other variables.This supplement extends that consideration to use of validation results from multiple set points within an application domain. The extension is the use of a multivariate metric. Multivariate metrics are designed to assess the capability of a model using experimental data and simulation results from more than one validation set point. This might be from multiple set points over space and/or time for a multidimensional case, for example. | Ballot #18-3672 for V&V and V&V20 approval closed January 14, 2019PINS #388Document rewritten in response to ballot commentsIn Fall 2022: VVUQ Standards Committee first consideration ballot #22-2843 was disapproved and comments were received. August 2023 Update: New First Consideration Ballot # 23-2311 was issued. | 75% | Standards Committee Ballot | First Edition 2023 | TR-18-2901B 23-2311 | VVUQ 20.2 | Regression of Validation Results | VVUQ 20.3 | Solution Verification of Unsteady Flow Calculations Supplement | March 2023 update: Plan to incorporate VVUQ 20.3 into the revision of V&V 20 – 2019. | Simulation at an Application Point | This supplement extends V&V 20 (2009) to use validation results from multiple set points within an application domain. The extension addressed is the use of validation results regressed to an application point within an application domain. An application point is commonly a set point at which a validation has not been performed. | Detailed outline drafted.PINS #389Defined working group | VVUQ 30 | Verification, Validation, and Uncertainty Quantification in Computational Simulation of Nuclear System Thermal Fluids Behavior | VVUQ 30.1-20XX | Scaling Methodologies for Prototypical Nuclear Power Systems Response - Guide | This Guide addresses the complexity of scaling effects and sub-scale validation experiments on a corresponding full-scale system and how they impact simulation validation. The focus of this Guide is on scaling methodologies for supporting the design of facilities capable of generating data that characterize the phenomena, determined to be high-ranked in phenomena identification and ranking (PIRT) studies for a transient of interest for both the entire plant [such facilities are known as integral effects test (IETs) facilities] and for components of the plant such as the nuclear core or the steam generator [such facilities are known as separate effects test (SETs) facilities]. | Comments from the VVUQ Standards Committee first consideration ballot comments were addressed.Update: VVUQ Standards Committee and VVUQ 30 Subcommittee #20-3805RC102 was approved with comments. Comments were addressed and clarifications were made in the draft, which will proceed to Public Review and Recirculation Ballot. | 90% | Standards Committee Approved | First Edition 2023 | TR 20-2804B 20-3805RC102 | Historical context of the subcommittee, seeking input from contributors in the field | Still in discussion | 1st Bench mark problem – manuscript underway from V&V Symposium 5/3/2017These examples will demonstrate the protocols used by the community | 2nd Bench mark problem V&V Symposium 2020 – Paper submission to JVVUQ | 3rd Benchmark Problem - Isothermal Single Jet Experiment and/or Non-Isothermal Single Jet Experiment within the Upper Plenum | VVUQ 40 | Verification, Validation, and Uncertainty Quantification in Computational Modeling of Medical Devices | V&V 40 | Assessing the Credibility of Computational Modeling through Verification and Validation: Application to Medical Devices | The scope of the Standard encompasses physics-based computational models used for medical device applications. This Standard augments other standards that present V&V methodologies, such as ASME V&V 10 and ASME V&V 20. Therefore, this Standard is intended for the practitioner who is familiar with V&V terminology. It does not present a method for incorporating user expertise or modeler pedigree, nor does it describe the specific V&V activities and rigor that are needed to establish credibility for a particular application and/or device. Instead, this Standard presents a framework for the practitioner to make that assessment using sound engineering judgment. This Standard is not a step-by-step guide, nor is it intended to present a quantitative method for establishing model credibility. While the framework was developed specifically for medical devices, the V&V 40 Subcommittee considers this Standard to be general enough to be applied to other disciplines. | Published November 19, 2018Plans for a revision of V&V 40 – 2018 are underway. See “VVUQ 40.6” below for future updates. | VVUQ 40.1 | Using (historical) clinical data as a comparator | PINS #701This item is in publication as a journal paper:Briant et al., “Use of real-world data for model credibility: Applications to medical device development”, Journal of Medical Devices 16, 031001-1-9, September 2022 | 100% | Technical Publication | Paper published 2022 | VVUQ 40.2 | Assessing Computational Model Credibility Using the ASME VVUQ 40 Risk-Based Framework:Tibial Tray Component Worst-Case Size Identification for Fatigue Testing Technical Report | Standards Committee Ballot 21-3306 for VVUQ 40.2 was disapproved by the VVUQ Standards Committee and comments were received. Update: First Consideration Ballot # 23-1383 was disapproved by the VVUQ Standards Committee and VVUQ 40 Subcommittee | 75% | Standards Committee Ballot | First Edition 2021 | TR 19-1006B23-1383 | VVUQ 40.3 | VVUQ for patient-specific models: surgical planning and clinical-decision making - software as a medical device | PINs #690A ballot is planned for the 3rd quarter of 2022 for Credibility assessment of patient-specific models following ASME V&V40.The following related journal publication was developed and submitted to Annals of Biomedical Engineering. Stott et al., “A critical comparison of comparators used for credibility evaluation of physics-based numerical spine models”. | 50% | Draft development | First Edition 2022 | VVUQ 40.4 | Verification best practicesCalculation Verification (Solids) | PINs #691The following technical publication was developed and submitted to the Journal of Verification, Validation, and Uncertainty Quantification.Guler et al., “Two calculation verification metrics used in the medical device industry: Revisiting the limitations of fractional change” | 90% | Technical Publication | 2022 | VVUQ 40.4 | Verification best practicesCode Verification (Fluids) | PINs #691 | 25% | Draft Development | First Edition 2021 | VVUQ 40.5 | Mock Submission – V&V 40 Practice in Regulatory Applications2 Technical Reports2 Documents FEA(50%) and CFD(10%) | PINS #718See VVUQ 40.X items in rows above. | Draft development | First Edition 2021 | VVUQ 40.6 | General methodology – Revisions to V&V 40 | Plans for a revision of V&V 40 – 2018 are underway.PINS #803 | VVUQ 50 | Verification, Validation, and Uncertainty Quantification of Computational Modeling for Advanced Manufacturing | VVUQ 50 | Verification, Validation, AND Uncertainty Quantification of Computational Modeling for Advanced Manufacturing | Procedures for verification, validation, and uncertainty quantification in modeling and computational simulation for advanced manufacturing. Four key areas they wish to develop to content: additive manufacturing, subtractive manufacturing, uncertainty in manufacturing, process control | PINS #674 issued July 2018. | Terminology, Concepts, Relationships and Taxonomy for VVUQ in Advanced Manufacturing. | Terminology task nearing completion, Concepts, Relationships & Taxonomy task after submission of Terminology reportEarly 2021 for review by committee | 75% | Draft Development | First Edition 2022 | VVUQ 50.1 Interactions with the Model Life Cycle | Completed 1) Survey relevant literature on managing models2) Define the Model Life CycleWorking on 3) Map requirements flow and activities for maintaining VVUQFuture4) Draft Material for proposed standardDraft review mid-2021Update: Ballot # 22-3095 was approved by the VVUQ 50 subcommittee and comments were received. | 50% | Subcommittee Ballot | First Edition 2022 | TR22-1467B22-3095 | VVUQ Methods in Data-driven and Hybrid models | Mission: provide a framework and guidance to the VV-UQ issues/problems related to data-driven and hybrid models that the manufacturing industry tackles. | 25% | Draft Development | VVUQ In-Process Technologies | 10% | Project Identified | VVUQ 60 | Verification, Validation, and Uncertainty Quantification of Computational Modeling in Energy Systems | VVUQ 60.1 | Considerations and Questionnaire for Simulation Software Selection An ASME Guideline Document | Update: VVUQ Standards Committee and VVUQ 60 Subcommittee #20-3805RC101 was approved with comments. | 90% | Standards Committee Approved | First Edition 2021 | TR 20-2507B 20-3732RC1 | VVUQ 60.2 | Guideline on Verification and Validation of Computational Modeling in Energy Systems | 10% | Draft Development | First Edition 2022 | VVUQ 70 | Verification, Validation, and Uncertainty Quantification of Machine Learning | Defining Terminology and Scoping | Project identified | VVUQ 80 | Verification, Validation, and Uncertainty Quantification in Computational Modeling of Pharmaceutical Products | VVUQ 90 | Verification, Validation, and Uncertainty Quantification in Computational Modeling of Airframe Structures | VVUQ 90 | Airframe Structure Modeling & Simulation Credibility Assurance Framework | Project Identified |
Note: Ballot (B) and Record (TR) numbers are system generated record numbers
Ballot Requirements
a) Other SC’s should be sent a review and comment ballot when the Subcommittee is sending out their documents for final Subcommittee ballots and/or
b) Other SC’s should be sent a review and comment ballot on all V&V standards committee first consideration ballots.
Note: (a) and (b) could occur concurrently.
Stages of Development
Project Identified
Draft Development or Revision Development
Draft Complete
Subcommittee Ballot
Standards Committee Ballot
Out for Public Review (ANSI or ASME Review)
Submitted for Publication
Published
% Complete Metrics
0-25%: Working group is defining scope, running analyses, expanding scope, re-doing analyses, etc
25-50%: Working group is putting together complete first draft of document
50-75%: Working group is revising first draft, but no review yet outside of working group
75-100%: Draft is in various stages of technical review, from full SC on up