Lightweight pedestrian detection utilizing multi-modal image fusion with dynamic multi-weight adjustment

To tackle the challenges associated with low pedestrian detection accuracy and the substantial model parameters in low-light conditions, this paper presents a lightweight multi-modal object detection network based on the YOLOv8 framework. The goal is to fully leverage the complementary characteristics of infrared and visible light images to enhance detection accuracy and robustness. Initially, dual feature extraction branches are established for infrared and visible light images, with modal weights determined adaptively based on the characteristics of the input images. This adaptive mechanism effectively leverages the complementary information from the distinct modalities, thereby enhancing detection accuracy. By designing a local illumination perception module to detect local illumination variations in input images, the fusion weights of the multimodal features are dynamically adjusted. This adjustment subsequently enhances detection performance under complex lighting conditions. Specially, to further enhance channel attention without introducing extra parameters, a parameter-free channel attention module has been developed. This module employs global max and average pooling to evaluate channel importance, optimizing fused channel features and enhancing recognition accuracy and robustness in complex scenes. Experiments on the LLVIP dataset demonstrate that our method achieves 98.1% mAP50 and 68.4% mAP at 238 FPS, requiring six to thirteen times fewer parameters than recent state-of-the-art methods while maintaining competitive accuracy, making it highly suitable for resource-constrained real-time applications.