High-Precision Trajectory Tracking for UAV Aerial Refueling Based on Deep Reinforcement Learning and Adaptive Model Predictive Control
High-Precision Trajectory Tracking for UAV Aerial Refueling Based on Deep Reinforcement Learning and Adaptive Model Predictive Control
Yan Gong
TLDR
A cascaded trajectory tracking and autonomous decision-making framework is proposed, integrating Soft Actor-Critic (SAC) deep reinforcement learning (DRL) with Adaptive Model Predictive Control (AMPC) with Markov Decision Process (MDP), achieving safe obstacle avoidance and optimal trajectory planning for the UAV in complex airflow environments.
Abstract
: To address the challenges of strong wake vortex aerodynamic disturbances, highly nonlinear dynamics of the hose-drogue system, and strict constraints during the unmanned aerial vehicle (UAV) aerial refueling docking phase, a cascaded trajectory tracking and autonomous decision-making framework is proposed, integrating Soft Actor-Critic (SAC) deep reinforcement learning (DRL) with Adaptive Model Predictive Control (AMPC). First, a six-degree-of-freedom (6-DOF) nonlinear dynamic model of the UAV incorporating the Burnham-Hallock wake vortex velocity field interference is established, alongside a lumped-mass catenary dynamic model for the hose-drogue system. Second, at the decision-making layer, an autonomous rendezvous agent based on the SAC algorithm is designed. By formulating a Markov Decision Process (MDP) featuring a continuous state-action space and a composite reward function, safe obstacle avoidance and optimal trajectory planning for the UAV in complex airflow environments are achieved. At the control layer, specifically targeting the precise docking phase, an AMPC is synthesized to enforce physical hard constraints on the actuators (i.e., deflection magnitude and rate limits). Furthermore, a first-order low-pass incremental state observer is introduced to estimate and compensate for the time-varying wind field disturbances induced by the wake vortex in real time. Finally, rigorous numerical evaluations and dynamic simulations are conducted for verification. The results demonstrate that, compared with traditional fuzzy PID and Nonlinear Dynamic Inversion (NDI) control, the proposed SAC-AMPC cascaded framework reduces the Root Mean Square Error (RMSE) along the X, Y, and Z axes by 61.9%, 61.1%, and 64.4%, respectively, relative to the fuzzy PID baseline. Additionally, the maximum transient overshoot is bounded within 0.55 m, the control energy consumption is reduced by 28.8%, and actuator