Information technology of multispectral video monitoring for human injuries detection and classification in emergency situations based on computer vision, convolutional neural networks and edge-computing
Information technology of multispectral video monitoring for human injuries detection and classification in emergency situations based on computer vision, convolutional neural networks and edge-computing
L. Chyrun,V. Vysotska,A. Chupryna
TLDR
The experimental results showed that using a multispectral approach increases the accuracy of human detection and injury classification to 96%, while ensuring stable system operation in difficult environmental conditions, confirming the promising application of multispectral sensors and deep learning models in creating intelligent systems to support rescue services and automate medical triage of victims.
Abstract
The article investigates approaches to automated identification of human injuries in emergency situations using multispectral video monitoring and deep learning. An intelligent system is proposed that combines visible and infrared spectral data to improve the accuracy of human detection and classification of injury types in complex environmental conditions, such as low light, smoke, or partial obstructions. The system architecture includes a multispectral video sensor module, data preprocessing and fusion modules, human detection algorithms, neural network models for feature extraction and injury classification, and a subsystem for assessing injury severity. Computer vision methods, convolutional neural networks, and edge-computing technology were used to implement data processing, which provides real-time video stream analysis without significant data transmission delays. Experimental studies were conducted on a multispectral set of images with different observation conditions and injury types. The experimental results showed that using a multispectral approach (RGB with IR) significantly increases the accuracy of human detection and damage classification compared to traditional RGB systems. The proposed model achieved an accuracy of 95,9 %, a Precision of 94,3 %, a Recall of 93,2 %, and an mAP of 0.92. The use of edge-computing technology enabled real-time video stream processing at over 25 frames per second and reduced data transmission latency to the central system. The results showed that using a multispectral approach increases the accuracy of human detection and injury classification to 96%, while ensuring stable system operation in difficult environmental conditions. The results confirm the promising application of multispectral sensors and deep learning models in creating intelligent systems to support rescue services and automate medical triage of victims.