ENG
РУС
Search
Menu
About the Edition
Open Access Statement
Editorial Board
Editorial Policy
Editorial Ethics
Rules for peer-review
Rules for presentation of papers
Forms of Submitted Documents
Recommendations for the Design of References to Our Journal
Editorial Staff
Information for Subscribers
Summaries of the Issue
List of Authors
Issues
Publishers license agreement
Procedure of work with personal data
Publications
2024
2023
2022
2021
2020
2019
2018
2017
2016
2015
2014
2013
2012
2011
2010
2009
2008
2007
2006
2005
2004
2003
2002
2001
Editor-in-Chief

Nikiforov
Vladimir O.
D.Sc., Prof.
Partners













































doi: 10.17586/2226-1494-2023-23-3-506-518

Facial keypoints detection using capsule neural networks

A. A. Boitsev, D. G. Volchek, E. N. Magazenkov, M. K. Nevaev, A. A. Romanov


Read the full article  ';
Article in Russian

For citation:
Boitsev A.A., Volchek D.G., Magazenkov E.N., Nevaev M.K., Romanov A.A. Facial keypoints detection using capsule neural networks. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2023, vol. 23, no. 3, pp. 506–518 (in Russian). doi: 10.17586/2226-1494-2023-23-3-506-518


Abstract
The problem of detecting key points of the face is investigated. This problem is quite relevant and important. The existing approaches of solving this problem, which are usually divided into parametric and nonparametric methods, are considered. As a result of the study, it was concluded that, nowadays, the most qualitative results are demonstrated by approaches based on deep learning methods. Two solutions are proposed: a capsule network with dynamic routing and a deep capsule network. The data for the experiments are 10,000 generated faces taken from Kaggle, marked up using MediaPipe. A method of using capsule architectures in neural networks to solve the problem of detecting key points of the face is proposed. The method includes the use of segmentation based on the key points of the face recognized using MediaPipe. Delaunay triangulation was used to build the face mesh. The architecture of a deep capsule network considering semantic segmentation was proposed. Based on the marked-up data, experiments on the detection of key points using the developed capsule neural networks were performed. According to the test results, the loss function reached values in range 2.50–2.90, the accuracy reached values in range 0.87–0.9. The proposed architecture can be used in technologies for comparing the geometry of the face grid of a real person with the geometry of the face grid of a three-dimensional model as well as in further studies of capsule neural networks by researchers in the field of image processing and analysis.

Keywords: capsule neural networks, detection of key points of the face, face image recognition, neural networks

References
  1. Volkova S.S., Matveev Yu.N. Convolutional neural networks for face anti-spoofing. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2017, vol. 17, no. 4, pp. 702–710. (in Russian). https://doi.org/10.17586/2226-1494-2017-17-4-702-710
  2. Dikiy D.I., Artemeva V.D. Research of artificial neural network applicability for user`s online handwritten signature verification. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2017, vol. 17, no. 4, pp. 677–684. (in Russian). https://doi.org/10.17586/2226-1494-2017-17-4-677-684
  3. Abiodun O.I., Kiru M.U., Jantan A., Omolara A.E., Dada K.V., Umar A.M., Linus O.U., Arshad H., Kazaure A.A., Gana U. Comprehensive review of artificial neural network applications to pattern recognition.IEEE Access, 2019,vol. 7,pp. 158820–158846. https://doi.org/10.1109/access.2019.2945545
  4. Krizhevsky A., Sutskever I., Hinton G.E. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2017,vol. 60, no. 6, pp. 84–90. https://doi.org/10.1145/3065386
  5. He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),2016,pp. 770–778. https://doi.org/10.1109/cvpr.2016.90
  6. Redmon J., Divvala S., Girshick R., Farhadi A. You only look once: Unified, real-time object detection. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 779–788. https://doi.org/10.1109/cvpr.2016.91
  7. Sabour S., Frosst N., Hinton G.E. Dynamic routing between capsules. Advances in Neural Information Processing Systems, 2017, vol. 30,pp. 3856–3866.
  8. Nguyen H.H., Yamagishi J., Echizen I. Capsule-forensics: Using capsule networks to detect forged images and videos. Proc. of the ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP),2019, pp. 2307–2311.https://doi.org/10.1109/icassp.2019.8682602
  9. Wang N., Gao X., Tao D., Yang H., Li X. Facial feature point detection: A comprehensive survey. Neurocomputing, 2018, vol. 275, pp. 50–65.https://doi.org/10.1016/j.neucom.2017.05.013
  10. Beham M.P., Roomi S.M.M. A review of face recognition methods. International Journal of Pattern Recognition and Artificial Intelligence, 2013, vol. 27, no. 4, pp. 1356005. https://doi.org/10.1142/S0218001413560053
  11. Li H., Lin Z.L., Brandt J., Shen X., Hua G. Efficient boosted exemplar-based face detection. Proc. of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 1843–1850.https://doi.org/10.1109/cvpr.2014.238
  12. Sun Y., Wang X., Tang X. Deep convolutional network cascade for facial point detection. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 3476–3483.https://doi.org/10.1109/cvpr.2013.446
  13. Zhang Z., Luo P., Loy C.C., TangX. Facial landmark detection by deep multi-task learning. Lecture Notes in Computer Science, 2014, vol. 8694, pp. 94–108.https://doi.org/10.1007/978-3-319-10599-4_7
  14. Trigeorgis G., Snape P., Nicolaou M.A., Antonakos E., ZafeiriouS. Mnemonic descent method: A recurrent process applied for end-to-end face alignment. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2016, pp. 4177–4187.https://doi.org/10.1109/cvpr.2016.453
  15. Zhu X., Lei Z., Liu X., Shi H., LiS.Z. Face alignment across large poses: A 3D solution. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2016, pp. 146–155.https://doi.org/10.1109/cvpr.2016.23
  16. Jourabloo A., Liu X. Large-pose face alignment via CNN-based dense 3D model fitting. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2016, pp. 4188–4196.https://doi.org/10.1109/cvpr.2016.454
  17. Kartynnik Y., Ablavatski A., Grishchenko I., Grundmann M. Real-time facial surface geometry from monocular video on mobile GPUs. arXiv,2019,arXiv:1907.06724. https://doi.org/10.48550/arXiv.1907.06724
  18. Wood E., Baltrušaitis T., Hewitt Ch., Johnson M., Shen J., Milosavljević N., Wilde D., Garbin S., Sharp T., Stojiljković I., Cashman T., Valentin J. 3D face reconstruction with dense landmarks. Lecture Notes in Computer Science, 2022, vol. 13673, pp. 160–177. https://doi.org/10.1007/978-3-031-19778-9_10
  19. Pechyonkin M. Understanding Hinton’s Capsule Networks. Part I: Intuition. Medium. 2018, December 18. Available at: https://medium.com/ai%C2%B3-theory-practice-business/understanding-hintons-capsule-networks-part-i-intuition-b4b559d1159b (accessed: 12.12.2022).
  20. Yu D., Wang H., Chen P., Wei Z. Mixed Pooling for convolutional neural networks. Lecture Notes in Computer Science, 2014, vol. 8818, pp. 364–375. https://doi.org/10.1007/978-3-319-11740-9_34
  21. Ding Y., Chen F., Zhao Y., Wu Z., Zhang C., Wu D. A stacked multi-connection simple reducing net for brain tumor segmentation. IEEE Access, 2019, vol. 7, pp. 104011–104024. https://doi.org/10.1109/access.2019.2926448


Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
Copyright 2001-2024 ©
Scientific and Technical Journal
of Information Technologies, Mechanics and Optics.
All rights reserved.

Яндекс.Метрика