Date of Award
6-2025
Document Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Department of Aeronautics and Astronautics
First Advisor
Donald L. Kunz, PhD
Abstract
The return of Great Power Competition to United States national security strategy necessitates renewed focus on a modern and more lethal military force. The United States Air Force has identified artificial intelligence and autonomy as key to delivering transformational strategic capabilities. This need, paired with the rapid advancement of machine learning, especially deep reinforcement learning (RL), led to questions concerning the optimality of reinforcement learning policies, the use of vehicle dynamics information in the creation of reinforcement learning rewards, and whether research conducted using environments with simple dynamics can be quickly and successfully scaled to more complex, more accurate dynamics, and eventually to flight. This document answers these questions by presenting a modular and hyper-configurable environment creation package known as CoRL, an implementation of single and multi-waypoint navigation subject to Dubins dynamics using CoRL, and a novel technique called DARB which incorporates dynamics information into the a priori balancing of rewards in multi-objective RL environments. The CoRL package and waypoint navigation environments were used to implement state-of-the-art RL techniques that successfully scaled to more complex environments with minimal re-work. These techniques trained a policy that flew the X-62 Vista. The DARB technique improved the proportion of successful episodes in tasks consisting of navigating to a single waypoint, where that waypoint is defined by a position, heading, and speed. DARB has no deleterious effect on sample efficiency, but does come with a small cost to time optimality. DARB is a simple method of improving the performance of an agent which removes the need for an ML engineer to iteratively and manually tune the gains of a complex reward function.
AFIT Designator
FY25-AFIT-ENY-DS-J-001
Recommended Citation
Merrick, Justin D., "Dynamics-Award Reward Balancing and Waypoint Navigation Using Deep Reinforcement Learning" (2025). Theses and Dissertations. 8375.
https://scholar.afit.edu/etd/8375
SF298 form of Merrick Dissertation FY25-AFIT-ENY-DS-J-001
Comments
A 12-month embargo was observed for posting this dissertation on AFIT Scholar.
Approved for public release. PA Case Number 88ABW-2026-0691
The SF298 form for this work is attached to this record page as an additional file.