Energy-Efficient Velocity Control With State Augmentation for Autonomous Vehicles

Velocity control in autonomous driving is an emerging technology that has achieved rapid progress over the last decade. However, existing velocity control studies ignore cascading disturbances in multi-lane scenarios and usually ignore the negative impact caused by harsh velocity decisions. To address these issues, we propose a reinforcement learning-based framework, called RISE (contRol velocIty for autonomouS vEhicle) to make velocity decisions for an autonomous vehicle in multi-lane traffic scenarios. To detect latent disturbances in the traffic flow, we propose a novel state encoder to learn the spatio-temporal correlation between different vehicles based on a well-designed impact graph. Afterward, we introduce an actor-critic paradigm to make velocity decisions with the aid of a hybrid reward function considering four optimization objectives: safety, efficiency, comfort, and impact. In particular, the impact term can penalize the harsh decisions of the autonomous vehicle, thus encouraging it to reduce the negative impacts on traffic flow. Further, we propose an improved RISE (RISE++) framework that incorporates a motion prediction model to augment state features for reasonable decisions, a modification of the reward function for energy efficiency, and a multi-worker paradigm for training efficiency. Extensive experiments offer evidence that the proposed framework can advance the state of the art in terms of effectiveness and efficiency.