Sum Rate Maximization in STAR-RIS-UAV-Assisted Networks: A CA-DDPG Approach for Joint Optimization

With the rapid advances in programmable materials, reconfigurable intelligent surfaces (RIS) have become a key technology for future wireless communications. The simultaneous transmitting and reflecting reconfigurable intelligent surfaces (STAR-RIS) can both transmit and reflect signals, thus extending signal coverage to full space and expanding application scenarios. This paper introduces an uncrewed aerial vehicle (UAV) to further enhance system flexibility and proposes an optimization design for the spectrum efficiency of the STAR-RIS-UAV-assisted wireless communication system. We present a deep reinforcement learning (DRL) algorithm capable of iteratively optimizing beamforming, phase shifts, and UAV positioning to maximize the system sum rate through continuous interactions with the environment. For deterministic policy-based DRL methods applied in wireless optimization, the exploration capability can be a limiting factor in discovering optimal policy. To overcome this issue, we introduce a stochastic perturbation factor to boost the exploration capability of the algorithm. With enhanced exploration, the algorithm’s capacity to accurately evaluate the state-action value function is necessary. On this basis, we propose a convolution-augmented deep deterministic policy gradient (CA-DDPG) algorithm by improving the exploration and evaluation mechanisms of the conventional DDPG framework to boost the system sum rate. The simulation results demonstrate that the CA-DDPG algorithm can adaptively interact with the environment to jointly optimize the decision variables, thus improving the system sum rate. In particular, the proposed algorithm achieves an average 29.1% improvement in system sum rate over the standard DDPG algorithm.