IMPROVE THE APPLICATION OF THE TD3 ALGORITHM IN LONGITUDINAL CONTROL FOR AUTONOMOUS DRIVING

Liang Xiong∗ and Yu Du∗

References

  1. [1] R.P. Borase, D.K. Maghade, S.Y. Sondkar, and S.N. Pawar,A review of PID control, tuning methods and applications,International Journal of Dynamics and Control, 9, 2021, 818–827.
  2. [2] L. Nie, J. Guan, C. Lu, H. Zheng, and Z. Yin, Longitudinalspeed control of autonomous vehicle based on a self-adaptivePID of radial basis function neural network, IET IntelligentTransport Systems, 12(6), 2018, 485–494.
  3. [3] L. Ge, Y. Zhao, and F. Ma, Towards longitudinal andlateral coupling control of autonomous vehicles using offsetfree MPC, Control Engineering Practice, 121, Apr. 2022,Art. no. 105074.
  4. [4] H. He, M. Han, W. Liu, J. Cao, M. Shi, N. Zhou, MPC-basedlongitudinal control strategy considering energy consumptionfor a dual-motor electric vehicle, Energy, 253, Aug. 2022, Art.no. 124004.
  5. [5] Y. Du, J. Chen, C. Zhao, C. Liu, and F. Liao, C.-Y. Chan,Comfortable and energy-efficient speed control of autonomousvehicles on rough pavements using deep reinforcement learning,Transportation Research Part C: Emerging Technologies, 134,Jan. 2022, Art. no. 103489.
  6. [6] Y. Yu, Y. Liu, J. Wang, N. Noguchi, and Y. He, Obstacleavoidance method based on double DQN for agricultural robots,Computers and Electronics in Agriculture, 204, Jan. 2023, Art.no. 107546.
  7. [7] S. Guo, X. Zhang, Y. Du, Y. Zheng, and Z. Cao, Path planningof coastal ships based on optimized DQN reward function,Journal of Marine Science and Engineering, 9(2), 2021, 210.
  8. [8] A. Bayuwindra, L. Wonohito, and B.R. Trilaksono, Designof DDPG-based extended look-ahead for longitudinal andlateral control of vehicle platoon, IEEE Access, 11, 2023,96648–96660.
  9. [9] J. Yang, W. Peng, and C. Sun, A learning control method ofautomated vehicle platoon at straight path with DDPG-basedPID, Electronics, 10(21), 2021, 2580.
  10. [10] B. Leng, Y.Z. Yu, M. Liu, X. Yang, and L. Xiong, Deepreinforcement learning-based drift parking control of automatedvehicles, Science China Technological Sciences, 66(4), 2023,1152–1165.
  11. [11] X. Wang, J. Zhang, D. Hou, and Y. Cheng, Autonomous drivingbased on approximate safe action, IEEE Transactions onIntelligent Transportation Systems, 24(12), 2023, 14320–14328.
  12. [12] J. Chen, S.E. Li, and M. Tomizuka, Interpretable end-to-end urban autonomous driving with latent deep reinforcementlearning, IEEE Transactions on Intelligent TransportationSystems, 23(6), 2022, 5068–5078.
  13. [13] Y. Gu, Z. Zhu, Y. Chu, J. Lv, X. Wang, and S. Xu, D3-TD3:Deep dense dueling architectures in TD3 algorithm for robotpath planning based on 3D point cloud, Journal of Circuits,Systems and Computers, 32(18), 2023, Art. no. 2350305.
  14. [14] W. Yuan, Y. Li, H. Zhuang, C. Wang, and M. Yang,Prioritized experience replay-based deep Q learning: Multiple-reward architecture for highway driving decision making, IEEERobotics & Automation Magazine, 28(4), 2021, 21–31.
  15. [15] J. Gao, W. Ye, J. Guo, and Z. Li, Deep reinforcement learningfor indoor mobile robot path planning, Sensors, 20(19), 2020,5493.
  16. [16] B. Wang, D. Zhao, and J. Cheng, Adaptive cruise control viaadaptive dynamic programming with experience replay, SoftComputing, 23(12), 2019, 4131–4144.
  17. [17] S. H. Kong, I.M.A. Nahrendra, and D.H. Paek, Enhanced off-policy reinforcement learning with focused experience replay,IEEE Access, 9, 2021, 93152–93164.
  18. [18] R. Cimurs and E. A. Merch´an-Cruz, Leveraging expertdemonstration features for deep reinforcement learning in floorcleaning robot navigation, Sensors, 22(20), 2022, 7750.
  19. [19] Y. Yun, D. Jeong, and S. Lim, Data-driven human-like cut-indriving model using generative adversarial network, ElectronicsLetters, 55(24), 2019, 1288–1290.
  20. [20] W. Kong, D. Zhou, Z. Yang, Y. Zhao, and K. Zhang, UAVautonomous aerial combat maneuver strategy generation withobservation error based on state-adversarial deep deterministicpolicy gradient and inverse reinforcement learning, Electronics,9(7), 2020, 1121.
  21. [21] J. Hu, X. Li, W. Hu, Q. Xu, and Y. Hu, Decision-makingfor connected and automated vehicles in challenging trafficconditions using imitation and deep reinforcement learning,International Journal of Automotive Technology, 24(6), 2023,1589–1602.
  22. [22] H. Liu and R. Jiang, Improving comfort level in traffic flowof CACC vehicles at lane drop on two-lane highways, PhysicaA: Statistical Mechanics and Its Applications, 575, Aug. 2021,Art. no. 126055.
  23. [23] A. Liu, T. Li, Y. Gu, and H. Dai, Cooperative extended stateobserver based control of vehicle platoons with arbitrarily smalltime headway, Automatica, 129, Jul. 2021, Art. no. 109678.
  24. [24] R. Hwang, H. Lee, and H.J. Hwang, Option compatible rewardinverse reinforcement learning, Pattern Recognition Letters,154, Feb. 2022, 83–89.
  25. [25] H. Xiang, Anti-disturbance adaptive control method of ACmotor servo on the basis of digital PID algorithm, MechatronicSystems and Control, 52(2), 2024, 130–139.
  26. [26] A. Kumar, Reinforcement learning: Application and advancestowards stable control strategies, Mechatronic Systems andControl, 51(1), 2023, 53–57.10

Important Links:

Go Back