# Research on Maneuver Decision of Unmanned Combat Aerial Vehicles Based on Segmented Reward Function and Improved Deep Q-Network

### Juntao Ruan, Juntao Ruan, Yi Qin, Fei Wang, Jianjun Huang, Fujie Wang, Fang Guo, Yaohua Hu

#### 2024

#### Abstract

Intelligent air combat is the main trend in the future, and the maneuvering decision-making ability of Unmanned combat aerial vehicles (UCAVs) affects the win-lose ending of the air battlefield. In order to study the problem of maneuver decision-making in UCAV 1V1 air combat, this paper proposes a maneuver decision generation algorithm based on the fusion of segmented reward function and improved deep Q-network. Firstly, this paper establishes a real mathematical model problem in the complex environment of air combat, and provides an equation expression that describes the spatial coordinate information, attitude information velocity information of the current state of UCAVs. This expression can provide basic maneuvering action instructions after passing the overload coefficient. Then, a segmented reward function was designed to guide unmanned aerial vehicles to develop towards their own advantages in the turn of aerial combat. Aiming at the problem of parameter uncertainty of deep Q-network (DQN) in maneuver decision-making process, an improved deep Q-network algorithm is proposed in the next. By introducing the extended Kalman filter (EKF), the uncertain parameter values of the strategy network are used to construct the system state equations, the parameters of the target network are used to construct the observation equations of the system, and the optimal parameter estimates of the DQN are obtained through the iterative updating solution of EKF. Simulation experiments show the effectiveness of the designed segmented reward function and the improved deep Q-network algorithm in autonomous maneuvering decision-making for UCAVs.

Download#### Paper Citation

#### in Harvard Style

Ruan J., Qin Y., Wang F., Huang J., Wang F., Guo F. and Hu Y. (2024). **Research on Maneuver Decision of Unmanned Combat Aerial Vehicles Based on Segmented Reward Function and Improved Deep Q-Network**. In *Proceedings of the 1st International Conference on Data Mining, E-Learning, and Information Systems - Volume 1: DMEIS*; ISBN 978-989-758-715-3, SciTePress, pages 94-101. DOI: 10.5220/0012911900004536

#### in Bibtex Style

@conference{dmeis24,

author={Juntao Ruan and Yi Qin and Fei Wang and Jianjun Huang and Fujie Wang and Fang Guo and Yaohua Hu},

title={Research on Maneuver Decision of Unmanned Combat Aerial Vehicles Based on Segmented Reward Function and Improved Deep Q-Network},

booktitle={Proceedings of the 1st International Conference on Data Mining, E-Learning, and Information Systems - Volume 1: DMEIS},

year={2024},

pages={94-101},

publisher={SciTePress},

organization={INSTICC},

doi={10.5220/0012911900004536},

isbn={978-989-758-715-3},

}

#### in EndNote Style

TY - CONF

JO - Proceedings of the 1st International Conference on Data Mining, E-Learning, and Information Systems - Volume 1: DMEIS

TI - Research on Maneuver Decision of Unmanned Combat Aerial Vehicles Based on Segmented Reward Function and Improved Deep Q-Network

SN - 978-989-758-715-3

AU - Ruan J.

AU - Qin Y.

AU - Wang F.

AU - Huang J.

AU - Wang F.

AU - Guo F.

AU - Hu Y.

PY - 2024

SP - 94

EP - 101

DO - 10.5220/0012911900004536

PB - SciTePress