Detection and Tracking of Search and Rescue Personnel under Hindered Light Conditions Using Hyperspectral Imaging

In the past two decades, advances in cost-effective fabrication techniques, miniature systems, intrinsic detectors and computing technology transformed hyperspectral imaging (HSI) from a bench-top scientific curiosity to a discipline with operational and fielded systems. Despite these advancements, there are still limitations such as degraded visual environment (DVE) conditions that reduce the visibility within a scene impacting HSI negatively. These conditions can be rain, haze or smoke and adversely affect the performance of the sensors by reducing their range of effectiveness in terms of detection, identification, and recognition. Additionally, benchmark detectors and classification techniques are mainly conducted on a pixel-wise basis which has proven to be a solid and effective technique but often isn't optimized for real-time detection of people or objects. This paper proposes a two-stage neural-net-based detector for real-time detection and tracking of firefighters subjected to DVEs using hyperspectral near-infrared (NIR) images. The proposed method first exploits the spatial features through object detection and then analyzes the extracted regions using a spectral-spatial pixel-wise algorithm to validate the presence of firefighters. The robustness of the proposed architecture is successfully demonstrated using various realistic scenarios.