IEEE AESS: Exploring Reinforcement Learning Techniques for Autonomous Vehicle Control Using the CARLA Simulator
Reinforcement learning has gained traction in research as a machine learning method for the control of autonomous vehicles. Its paradigm of learning by traversing through a series of states has shown to work well with the dynamic nature of driving. However, for safety reasons, much of the research effort is performed in simulators. This work sought to utilize a driving simulator to train an autonomous vehicle using various reinforcement learning methods, with the goal of the vehicle to reach a destination while staying within the road and avoiding collisions with other obstacles.
Three methods were explored. The first utilized a Deep Q-Network (DQN) based algorithm with the state being defined by direct values and reward shaping. The second method extended upon the first, utilizing a Proximal Policy Optimization (PPO) based algorithm. It included additions to the reward function based on lessons learned and experimentation with a CNN based state representation. It was shown to learn to solve a driving scenario with less needed steps, as compared to the first method. The final method utilized behavioral cloning with images as its state representation. It was shown to be a minor improvement over its baseline.
Talk is limited to US citizens. Registration is required for non-SwRI employees.
Cookies and drinks will be provided.
Date and Time
Location
Hosts
Registration
- Date: 18 Jun 2025
- Time: 12:00 PM to 01:00 PM
- All times are (UTC-05:00) Central Time (US & Canada)
-
Add Event to Calendar
- 6220 Culebra Rd
- San Antonio, Texas
- United States 78238
- Building: Building 51
- Starts 04 March 2025 12:00 AM
- Ends 16 June 2025 05:00 PM
- All times are (UTC-05:00) Central Time (US & Canada)
- No Admission Charge
Speakers
Joseph Clemmons
Reinforcement Learning
Reinforcement learning has gained traction in research as a machine learning method for the control of autonomous vehicles (AV). Its paradigm of learning by traversing through a series of states has shown to work well with the dynamic nature of driving. However, for safety reasons, much of the research effort is performed in simulators. Among those simulators, the Cars Learn to Act (CARLA) driving simulator has gained popularity due to its realistic physics engine. This work sought to utilize the CARLA driving simulator to train an autonomous vehicle (known as the ego vehicle) using various reinforcement learning methods. The goal of the ego vehicle was to reach a destination, while staying within the road and avoiding collisions with other obstacles.
Three methods were explored. The first utilized a Deep Q-Network (DQN) based algorithm with the state being defined by direct values and reward shaping. It was shown that the trained ego vehicle could learn to reach its destination with 5000 episodes for all scenarios tested.
The second method extended upon the first, utilizing a Proximal Policy Optimization (PPO) based algorithm. It included additions to the reward function based on lessons learned and experimentation with a CNN based state representation. It was shown to learn to solve a driving scenario with less needed steps, as compared to the first method.
The final method utilized behavioral cloning with images as its state representation. It was shown to be a minor improvement over its TCP baseline.
Biography:
Joseph Clemmons received a B.S. in Electrical Engineering in 2022 and an M.S. in Electrical and Computer Engineering in 2024 from the University of Texas at San Antonio. He is currently an engineer in the Defense and Intelligence Solutions Division at Southwest Research Institute under the SigInt Solutions Department. His research interests include machine learning, image processing, and digital signal processing.
Address:Texas, United States