Validation of Discrete and Functional Simulation Responses Over Experimental Regions Using Response Surfaces

Scott M. Storm


The Director of Test and Evaluation has placed great emphasis on the use of experimental design to properly collect and analyze data from military assets. This emphasis on statistical based data collection plays a large role in the validation of simulation models. It is prudent to validate simulation models across these experimental regions to ensure accuracy at all desirable configurations, to include points within the region where no experimental data has been collected. This research proposes the use of response surfaces to capture the behavior of the response across the experimental region. The first paper develops a methodology for point-wise validation of model response output at unique points within an experimental region using regression based tolerance intervals. The second paper develops an F-statistic for validation of the simulation model over the entire region based on a ratio of mean square errors constructed from the model and system data with regards to a common response surface. The third paper proposes an extension to the Correlation Analysis (CORA) objective rating system that uses functional regression techniques to allow for assessments of functional response data when no system data is available. The fourth paper applies each methodology to the Sandia thermal challenge problem.