Fast labeling pipeline approach for a huge aerial sensed dataset

Fedulin A.M., Voloshina N.V. Fast labeling pipeline approach for a huge aerial sensed dataset. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2024, vol. 24, no. 2, pp. 190–197. doi: 10.17586/2226-1494-2024-24-2-190-197

Modern neural network technologies are actively used for Unmanned Aerial Vehicles (UAVs). Convolutional Neural Networks (CNN), are mostly used for object detection, classification, and tracking tasks, for example, for such objects as fires, deforestations, buildings, cars, or people. However, to improve effectiveness of CNNs it is necessary to perform their fine-tuning on new flight data periodically. Such training data should be labeled, which increases total CNN fine- tuning time. Nowadays, the common approach to decrease labeling time is to apply auto-labeling and labeled objects tracking. These approaches are not effective enough for labeling of 8 hours’ huge aerial sensed datasets that are common for long-endurance USVs. Thus, reducing data labeling time is an actual task nowadays. In this research, we propose a fast aerial data labeling pipeline especially for videos gathered by long-endurance UAVs cameras. The standard labeling pipeline was supplemented with several steps such as overlapped frames pruning, final labeling spreading over video frames. The other additional step is to calculate a Potential Information Value (PIV) for each frame as a cumulative estimation of frame anomality, frame quality, and auto-detected objects. Calculated PIVs are used than to sort out frames. As a result, an operator who labels video gets informative frames at the very beginning of the labeling process. The effectiveness of proposed approach was estimated on collected datasets of aerial sensed videos obtained by long-endurance UAVs. It was shown that it is possible to decrease labeling time by 50 % in average in comparison with other modern labeling tools. The percentage of average number of labeled objects was 80 %, with them being labeled for 40 % of total pre-ranged frames. Proposed approach allows us to decrease labeling time for a new long-endurance flight video data significantly. This makes it possible to speed up neural network fine-tuning process. As a result, it became possible to label new data during the inter-flight time that usually takes about two or three hours and is too short for other labeling instruments. Proposed approach is recommended to decrease UAVs operators working time and labeled dataset creating time that could positively influence on the time necessary for the fine-tuning a new effective CNN models.

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License