Toward Automated Instructor Pilots in Legacy Air Force Systems: Physiology-based Flight Difficulty Classification via Machine Learning
The United States Air Force (USAF) is struggling to train enough pilots to meet operational requirements. Technology has advanced rapidly over the last 70 years but USAF pilot training has not. Modern operational requirements demand a change and, for this reason, USAF senior leadership has advocated for innovation. The automation of instructor and evaluator pilots in select bottlenecks (e.g., simulators) is one such measure. However, to implement this vision, numerous technical issues must be mitigated. Accurate classification of flight difficulty is a foundational problem underpinning many of these technical issues, which requires either the acquisition of new systems or the development of new procedures. Therefore, given this need and the costly nature of purchasing new equipment, physiological-based classification of flight difficulty is our focus herein. Leveraging multimodal data from a designed experiment of pilots landing a simulated aircraft, we develop a high-quality machine learning pipeline for classifying flight difficulty, called the Multi-Modal Functional-based Decision Support System (MMF-DSS). MMF-DSS distills a tabular set of features from our multimodal and functional data through the use of functional principal component analysis, summary statistics, and BorutaSHAP. In this manner, information is derived from the time-series data via the generation of hundreds of features, of which a small subset having the most predictive capability is discerned. Four full factorial designs are used to perform hyperparameter tuning on a set of classifiers. In so doing, a superlative technique is identified. Impacts on executive decision making are examined as well as associated policymaking implications. Alternative classifiers are considered for use within our pipeline that trade predictive accuracy for cost efficiency, and recommendations for choosing among these alternatives is provided.
Expert Systems with Applications
Caballero, W. N., Gaw, N., Jenkins, P. R., & Johnstone, C. (2023). Toward automated instructor pilots in legacy Air Force systems: Physiology-based flight difficulty classification via machine learning. Expert Systems with Applications, 231(30 November), 120711. https://doi.org/10.1016/j.eswa.2023.120711