Deep Learning based Object Detection and Tracking for Maritime Situational Awareness

Improving real-time situational awareness using deep-learning based video processing is of great interest in maritime and inland waterway environments. For instance, automating visual analysis for the classification and interpretation of the objects surrounding a vessel remains a critical challenge towards more autonomous navigational system. The complexity dramatically increases when we address waterway environments with a more dense traffic compared to open sea, and presenting navigation marks that need to be detected and correctly understood to take correct decisions. In this paper, we will therefore propose a new training dataset tailored to the navigation and mooring in waterway environments. The dataset contains 827 representative images gathered in various Belgian waterways. The images are captured on board a navigating barge and from a camera mounted on a drone. The dataset covers a range of realistic conditions of traffic and weather conditions. We investigate in the current study the training of the YOLOv5 model for the detection of seven different classes corresponding to vessels, obstacles and different navigation marks. The detector is combined with a pretrained Deep Sort Tracker. The YOLOv5 training results proved to reach an overall mean average precision of 0.891 for an intersection over union of 0.5.