Reinforcement Learning for Team Based Air Combat Maneuvering Decisions with Directed Energy Weaponry

Author

Joshua D. Combs

Date of Award

3-2024

Document Type

Thesis

Degree Name

Master of Science in Operations Research

Department

Department of Operational Sciences

First Advisor

Matthew Robbins, PhD

Abstract

Leveraging the Advanced Framework for Simulation, Integration, and Modeling (AFSIM) we investigate the use of reinforcement learning (RL) techniques for imbuing AUCAV agents with high-quality behaviors for the within-visual-range air combat maneuvering problem (ACMP). We formulate the 2v2 WVR ACMP as a Markov decision process wherein friendly AUCAVs are equipped with DEW capabilities and operate with 6 degrees of freedom. We utilize the Double Deep Q-Network RL algorithm, which centrally trains two friendly AUCAVs and employ a phased learning approach, initially exposing the AUCAVs to a dense reward environment for early training, followed by a sparse reward environment to encourage emergent behaviors. An illustrative experiment is designed to assess combat performance of AUCAVs, which can be used to inform future research endeavors. Qualitative analysis of learned combat maneuvers and quantitative experiments on different DEW weapon parameters offer insights into the efficacy of our RL solution procedure and its potential for informing the development of future weapon concepts.

AFIT Designator

AFIT-ENS-MS-24-M-076

Comments

A 12-month embargo was observed for posting this work on AFIT Scholar.

Distribution Statement A, Approved for Public Release. PA case number on file.

Recommended Citation

Combs, Joshua D., "Reinforcement Learning for Team Based Air Combat Maneuvering Decisions with Directed Energy Weaponry" (2024). Theses and Dissertations. 7711.
https://scholar.afit.edu/etd/7711