A Model-based DNN for Learning Hybrid Beamforming in Terahertz Massive MIMO Systems

Hybrid analog and digital beamforming (HBF) is an essential technique for terahertz (THz) communications to support high spectral efficiency with affordable cost. Optimizing hybrid beamforming with deep learning can improve system performance and enhance robustness to imperfect channels. However, pure data-driven neural networks suffer from high training complexity and weak interpretability, while the performance of existing model-based approaches (e.g., deep unfolding) is limited by the algorithm itself. In this paper, we propose a model-based neural network, namely HBF-NN, to optimize hybrid beamforming for multi-antenna multi-carrier THz systems, which consists of two jointly trained modules for optimizing analog and digital beamforming matrices, respectively. To simplify the function to be learned, we propose to optimize the analog beamforming in angle domain. To learn the digital beamforming efficiently, we conceive a graph neural network structure by harnessing the permutation property, recursive property, and the structure of a commonly-used algorithm of singular value decomposition. Simulation results show that the proposed HBF-NN achieves higher spectral efficiency than numerical algorithms, while requiring significantly fewer training samples, free parameters, and less training time than existing data-driven counterpart to achieve the same performance.