Real-time Automated Aerial Refueling Using Stereo Vision

Document Type

Conference Proceeding

Publication Date



Aerial Refueling provides the United States Air Force with aircraft endurance to maintain air superiority and aid operations around the world. With the emergence of Unmanned Aerial Vehicles (UAVs), air operations can be done more efficiently and ground operations gain increased support as a result of UAV readiness and endurance. UAVs currently lack the capability to perform mid-flight aerial refueling due to command and control signal. To mitigate delay, an automated system must relay an approach vector or rel-nav solution directly between the tanker and UAV for safe refueling operations. To properly test and evaluate the rel-nav solution, simulations and flight tests must be performed to verify and validate accuracy. Our research is twofold. First, we present a three dimensional geometrically accurate virtual world providing real time testing of Automated Aerial Refueling algorithms and methods. The virtual world allows researchers to deterministically generate sensor data and operate on it while simultaneously visualizing the output. Second, we present a computer vision approach to the automated aerial refueling problem which consumes the aforementioned output in real time. Our approach uses Open Computer Vision libraries to estimate position and attitude using synthetic stereo camera feeds mounted on a virtual KC-46. The virtual world enables rapid testing of our tanker-receiver relative position estimation in real time. This implementation will be flown to support a live flight test scheduled for late 2017.


Published by Springer as a work of the U.S. Federal government. Its text may be subject to foreign copyright protection.

The rights of this work are transferred to the extent transferable according to title 17 U.S.C. 105. © Springer International Publishing AG 2016

The "Link to Full Text" on this page loads the PDF of the work, hosted at Springer Nature.



Source Publication

Advances in Visual Computing. ISVC 2016 (LNCS 10073)