Improved YOLOv11 for Multi-Modal Human Detection in Dense Smoke Environments

In fire rescue scenarios, dense smoke severely degrades the performance of traditional human detection systems due to reduced contrast and loss of fine details. To address these challenges, this paper presents an improved YOLOv11 framework, termed YOLOv11-HYSZ, incorporating multi-modal data fusion to enhance human target detection in smoke-obscured environments. The proposed method integrates infrared and thermal imaging data with visible spectrum information by expanding the input channels and introducing a dynamic channel segmentation module, termed the SilenceChannel, to effectively process heterogeneous sensor inputs. In addition, a multi-modal feature fusion strategy based on cross-modal attention is adopted in the mid-level layers of the network to leverage complementary features from different modalities. To further boost the performance of the system, a joint optimization approach combining dehazing and detection is employed, where a de-smoke algorithm based on a deep segmentation network refines the input images and enhances target features before detection. Extensive experiments conducted on both augmented datasets and real-world smoke scenarios demonstrate that the proposed method significantly improves detection accuracy and robustness, achieving higher precision and recall rates compared to conventional YOLO models under challenging conditions.