An optimized real-time human detected keyframe extraction algorithm (HDKFE) based on faster R-CNN

Abstract

The aim of this project is to support criminal investigators by utilizing surveillance camera footage in their investigations.  To apprehend the culprit, it is necessary to examine the video footage and extract the relevant and crucial information. Analyzing lengthier videos might provide challenges due to the time needed to process the entire video while maintaining its semantic features. In this situation, a dataset is collected in real-time to aid in the criminal investigation, which consequently requires the use of keyframes. The study illustrates that Content-based video retrieval (CBVR) enables the video analysis technique. Keyframe extraction is a significant component of video analysis. The main objective of key frame extraction is to reduce the amount of repetitive frames in a video, thereby improving the clarity and efficiency of the scenario. Moreover, it optimizes video sequences to expedite processing. The study paper introduces the Human Detected Keyframe Extraction algorithm (HDKFE), which utilizes a dual-stage methodological approach. The Faster Region-Convolutional Neural Network (Faster R-CNN) detects humans in surveillance by identifying frames that contain humans and reporting them using an optimized threshold value. The frames then identify a suitable keyframe by recognizing local maxima through the absolute difference between frames in the subsequent phase. This significantly decreases the complexity of long-term criminal investigations. The experimental report reveals that the HDKFE approach achieves a precision of 98.87% while minimizing both space and time complexity.