Manifold Learning-based Frequency Estimation for extracting ENF signal from digital video

• Main Authors: Jeon, Youngbae, Han, Hyekyung, Yoon, Ji Won
• Format: Conference Proceeding
• Language: English
• Subjects: Forensics; Frequency conversion; Frequency estimation; Manifold learning; Manifolds; Pattern recognition
• ISSN: 2831-7475
• DOI: 10.1109/ICPR56361.2022.9956558

Using electrical network frequency (ENF) for video forensics has been intensely studied in recent years. The ENF signal found in videos has twice the electrical frequency (100 Hz or 120 Hz), whereas frame rates of common videos are relatively low (around 30 Hz). To extract ENF signal from video, state-of-the-art works exploit the rolling shutter effect. However, this method has a constraint that the region affected by the flickering light has to be large enough to contain all the information which light leaves at the pixels. As these regions are only part of the scene in many cases, it is hard to take advantage of the rolling shutter effect. In this paper, we propose a novel method to extract ENF signals by topological approach without utilizing the rolling shutter effect. Based on the fact that the topological representation of the possible outcomes is in the form of a closed-loop, we obtain the phase angles of each frame using manifold learning. We convert the phase angles into the frequency values based on the prior knowledge about the nominal frequency of ENF and the frame rate of the video. We tested two different manifold learning algorithms (i.e., UMAP and t-SNE) and compared the result with the state-of-the-art works, and t-SNE shows the best performance achieving root-mean-square error (RMSE) of 0.00036 Hz.