Video Deepfake Detection Using EfficientNet and LSTM

Abstract

As the prevalence of advanced face manipulation technologies continues to grow, the detection of deepfakes has emerged as a pivotal area of research, presenting ethical and security challenges. This study introduces an enhanced deepfake detection framework that integrates the EfficientNetB0 model, cutting-edge convolutional neural network (CNN) architecture, with an added long short-term memory (LSTM) layer. The CNN component, EfficientNetB0, excels at extracting spatial features from individual frames, capturing fine-grained details and artifacts indicative of manipulation. The LSTM layer processes sequential dependencies across frames, leveraging temporal inconsistencies that are often present in deepfake videos but difficult to detect from individual images alone. By combining spatial and temporal feature learning, this hybrid CNN-LSTM approach enhances the system’s ability to distinguish real from manipulated media with greater accuracy. Leveraging video frame extraction and comprehensive data augmentation techniques, the system preprocesses inputs to improve generalization on limited training data. The EfficientNetB0 model, pre-trained on the ImageNet dataset, serves as the backbone for feature extraction, utilizing its highly efficient architecture, which includes depth wise separable convolutions. An evaluation on the Celeb-DF dataset demonstrates that the proposed system maintains high accuracy and robustness in detecting deepfake content, while preserving computational efficiency, making it well-suited for real-world applications. The experimental results validate the effectiveness of this extended approach, emphasizing its potential to significantly contribute to the mitigation of the detrimental impacts of deepfakes.

Country : India

1 Satwika.M2 Neha.K3 Pranavya.A4 Rishika.K5 Siva Sankar Namani

  1. Department of CSE (AI & ML), G. Narayanamma Institute of Technology and Science, Hyderabad, Telangana, India
  2. Department of CSE (AI & ML), G. Narayanamma Institute of Technology and Science, Hyderabad, Telangana, India
  3. Department of CSE (AI & ML), G. Narayanamma Institute of Technology and Science, Hyderabad, Telangana, India
  4. Department of CSE (AI & ML), G. Narayanamma Institute of Technology and Science, Hyderabad, Telangana, India
  5. Assistant Professor, Department of CSE (AI & ML), G. Narayanamma Institute of Technology and Science, Hyderabad, Telangana, India

IRJIET, Volume 9, Issue 3, March 2025 pp. 148-154

doi.org/10.47001/IRJIET/2025.903018

References

  1. Agarwal S, Farid H, Fried O, and Agrawala M (2020) Detecting Deep-Fake Videos from Phoneme-Viseme Mismatches. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA.
  2. Amerini I, Galteri L, Caldelli R, and Del Bimbo A (2019) Deepfake Video Detection through Optical Flow Based CNN. In: IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea (South).
  3. Ding X, Raziei Z, Larson E.C. et al. (2020).
  4. Dolhansky B, Howes R, Pflaum B, Baram N, Ferrer C (2019) The Deepfake Detection Challenge (DFDC) Preview Dataset. arXiv:1910:08854.
  5. Fei J, Xia Z, Yu P, Xiao F (2020) Exposing AI-generated videos with motion magnification. Multimedia Tools Appl. 80(20), 30789–30802.
  6. Fernando T., Fookes C., Denman S. and Sridharan S. (2021) Detection of Fake and Fraudulent Faces via Neural Memory Networks. In: IEEE Transactions on Information Forensics and Security, vol. 16, pp. 1973-1988, doi: 10.1109/TIFS.2020.3047768.
  7. Hashmi M.F., Ashish B.K.K., Keskar A.G., Bokde N.D., Yoon J.H., Geem Z.W. (2020) An Exploratory Analysis on Visual Counterfeits Using Conv-LSTM Hybrid Architecture. IEEE Access 8, 101293–101308.
  8. Hsu C.-C., Zhuang Y-Xiu., Lee C.-Y. (2020) Deep fake image detection based on pairwise learning. Applied Sciences 10(1), 370.
  9. Huy H. Nguyen, Fuming Fang, Junichi Yamagishi, Isao Echizen (2019) Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos. In: IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).
  10. Komal Chugh, Parul Gupta, Abhinav Dhall, and Ramanathan Subramanian (2018) Not made for each other – Audio-Visual Dissonance-based Deepfake Detection and Localization. In Woodstock ’18: ACM Symposium on Neural Gaze Detection, June 03–05.
  11. Li, Haodong; Li, Bin; Tan, Shunquan; Huang, Jiwu: Identification of deep network generated images using disparities in color components. Signal Process. (2020).
  12. Li, Y.; Chang, M.; Lyu, S. (2018) In Ictu Oculi Exposing AI Created Fake Videos by Detecting Eye Blinking. In: IEEE International Workshop on Information Forensics and Security (WIFS), Hong Kong, Hong Kong.
  13. Li Y, Lyu S (2018) Exposing Deep Fake Videos By Detecting Face Warping Artifacts. In: IEEE Conference Computer. Vision Pattern Recognition.
  14. L. Zhang, T. Qiao, M. Xu, N. Zheng and S. Xie (2023) Unsupervised Learning-Based Framework for Deepfake Video Detection. In: IEEE Transactions on Multimedia, vol. 25, pp. 4785-4799, doi: 10.1109/TMM.2022.3182509.
  15. Matthews T.F., Cootes J.A., Bangham S.C., Harvey R. (2002) Extraction of visual features for lip reading. IEEE Trans. Pattern Anal. Mach. Intell. 24(2), 198–213.
  16. Matern F, Riess C, Stamminger M (2019) Exploiting Visual Artifacts to Expose Deep fakes and Face Manipulations. In: IEEE Winter Applications of Computer Vision Workshops (WACVW), Waikoloa Village, HI, USA.
  17. Minh Dang L., Hassan S.I., Im S., Moon H. (2019) Face image manipulation detection based on a convolutional neural network. Expert Syst. Appl. 129, 156–168.
  18. Mingxing Tan, Quoc V. Le (2019) EfficientNet: Re-thinking Model Scaling for Convolutional Neural Networks. Cornell University.
  19. Montserrat D.M. et al. (2020) Deepfakes Detection with Automatic Face Weighting In: IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA.
  20. N. Qazi and I. Ahmed (2024) Enhancing Authenticity Verification with Transfer Learning and Ensemble Techniques in Facial Feature-Based Deepfake Detection. In: 2024 14th International Conference on Pattern Recognition Systems (ICPRS), London, United Kingdom, pp. 1-6, doi: 10.1109/ICPRS62101.2024.10677831.
  21. Prashnani E., Goebel M. and Manjunath B. S. (2025) Generalizable Deepfake Detection with Phase-Based Motion Analysis. In: IEEE Transactions on Image Processing, vol. 34, pp. 100-112, doi: 10.1109/TIP.2024.3441821.
  22. Rössler A, Cozzolino D, Verdoliva L, Riess C, Thies J, Nießner M. (2018) FaceForensics: A large-scale video dataset for forgery detection in human faces.
  23. Rössler A, Cozzolino D, Verdoliva L, Riess C, Thies J, Niessner M. (2019) FaceForensics++: learning to detect manipulated facial images. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV).
  24. Sabir E, Cheng J, Jaiswal A, Almageed W.A., Masi I, Natarajan P. (2019) Recurrent Convolutional Strategies for Face Manipulation Detection in Videos. IEEE Conf. Comput. Vision Pattern Recogn. 3, 80–87.
  25. S. J. Pipin, R. Purba and M. F. Pasha (2022) Deepfake Video Detection Using Spatiotemporal Convolutional Network and Photo Response Non Uniformity. In: 2022 IEEE International Conference of Computer Science and Information Technology (ICOSNIKOM), Laguboti, North Sumatra, Indonesia, pp. 1-6, doi: 10.1109/ICOSNIKOM56551.2022.10034890.
  26. S. R. Adnan and H. A. Abdulbaqi (2022) Deepfake Video Detection Based on Convolutional Neural Networks. In: 2022 International Conference on Data Science and Intelligent Computing (ICDSIC), Karbala, Iraq, pp. 65-69, doi: 10.1109/ICDSIC56987.2022.10075830.
  27. Torfi A, Iranmanesh S.M., Nasrabadi N., Dawson J. (2017) 3D Convolutional Neural Networks for Cross Audio-Visual Matching Recognition. IEEE Access 5, 22081–22091.
  28. T. Fernando, C. Fookes, S. Denman and S. Sridharan (2021) Detection of Fake and Fraudulent Faces via Neural Memory Networks. In: IEEE Transactions on Information Forensics and Security, vol. 16, pp. 1973-1988, doi: 10.1109/TIFS.2020.3047768.
  29. Tran V. -N., Kwon S. -G., Lee S. -H., Le H. -S. and Kwon K. -R. (2023) Generalization of Forgery Detection With Meta Deepfake Detection Model. In: IEEE Access, vol. 11, pp. 535-546, doi: 10.1109/ACCESS.2022.3232290.
  30. Y. Patel et al. (2023) An Improved Dense CNN Architecture for Deepfake Image Detection. In: IEEE Access, vol. 11, pp. 22081-22095, doi: 10.1109/ACCESS.2023.3251417.
  31. Z. Guo, G. Yang, J. Chen and X. Sun (2023) Exposing Deepfake Face Forgeries With Guided Residuals. In: IEEE Transactions on Multimedia, vol. 25, pp. 8458-8470, doi: 10.1109/TMM.2023.3237169.
  32. Z. Zhang, C. Mal, B. Ding and M. Gao (2020) Detecting Manipulated Facial Videos: A Time Series Solution. In: 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy, pp. 2817-2823, doi: 10.1109/ICPR48806.2021.9412610.