Date of Award
3-2022
Document Type
Thesis
Degree Name
Master of Science
Department
Department of Mathematics and Statistics
First Advisor
Christine M. Schubert Kabban, PhD
Abstract
Artificial neural networks (ANNs) are popular tools for accomplishing many machine learning tasks, including predicting continuous outcomes. However, the general lack of confidence measures provided with ANN predictions limit their applicability, especially in military settings where accuracy is paramount. Supplementing point predictions with prediction intervals (PIs) is common for other learning algorithms, but the complex structure and training of ANNs renders constructing PIs difficult. This work provides the network design choices and inferential methods for creating better performing PIs with ANNs to enable their adaptation for military use. A two-step experiment is executed across 11 datasets, including an imaged-based dataset. Two non-parametric methods for constructing PIs, bootstrapping and conformal inference, are considered. The results of the first experimental step reveal that the choices inherent to building an ANN affect PI performance. Guidance is provided for optimizing PI performance with respect to each network feature and PI method. In the second step, 20 algorithms for constructing PIs—each using the principles of bootstrapping or conformal inference—are implemented to determine which provides the best performance while maintaining reasonable computational burden. In general, this trade-off is optimized when implementing the cross-conformal method, which maintained interval coverage and efficiency with decreased computational burden.
AFIT Designator
AFIT-ENC-MS-22-M-001
DTIC Accession Number
AD1166827
Recommended Citation
Contarino, Alexander N., "Constructing Prediction Intervals with Neural Networks: An Empirical Evaluation of Bootstrapping and Conformal Inference Methods" (2022). Theses and Dissertations. 5315.
https://scholar.afit.edu/etd/5315