ENG
РУС
Search
Menu
About the Edition
Open Access Statement
Editorial Board
Editorial Policy
Editorial Ethics
Rules for peer-review
Rules for presentation of papers
Forms of Submitted Documents
Recommendations for the Design of References to Our Journal
Editorial Staff
Information for Subscribers
Summaries of the Issue
List of Authors
Issues
Publishers license agreement
Procedure of work with personal data
Publications
2024
2023
2022
2021
2020
2019
2018
2017
2016
2015
2014
2013
2012
2011
2010
2009
2008
2007
2006
2005
2004
2003
2002
2001
Editor-in-Chief

Nikiforov
Vladimir O.
D.Sc., Prof.
Partners













































doi: 10.17586/2226-1494-2023-23-4-720-733

Attacks based on malicious perturbations on image processing systems and defense methods against them

D. A. Esipov, A. Y. Buchaev, A. Kerimbay, Y. V. Puzikova, S. K. Saidumarov, N. S. Sulimenko, I. Y. Popov, N. S. Karmanovskiy


Read the full article  ';
Article in Russian

For citation:
Esipov D.A., Buchaev A.Y., Kerimbay A., Puzikova Y.V., Saidumarov S.K., Sulimenko N.S., Popov I.Yu., Karmanovskiy N.S. Attacks based on malicious perturbations on image processing systems and defense methods against them. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2023, vol. 23, no. 4, pp. 720–733 (in Russian). doi: 10.17586/2226-1494-2023-23-4-720-733


Abstract
Systems implementing artificial intelligence technologies have become widespread due to their effectiveness in solving various applied tasks including computer vision. Image processing through neural networks is also used in security- critical systems. At the same time, the use of artificial intelligence is associated with characteristic threats including disruption of machine learning models. The phenomenon of triggering an incorrect neural network response by introducing perturbations that are visually imperceptible to a person was first described and attracted the attention of researchers in 2013. Methods of attacks on neural networks based on malicious perturbations have been continuously improved, ways of disrupting the operation of neural networks in processing various types of data and tasks of the target model have been proposed. The threat of disrupting the functioning of neural networks through these attacks has become a significant problem for systems implementing artificial intelligence technologies. Thus, research in the field of countering attacks based on malicious perturbations is very relevant. This article describes current attacks, provides an overview and comparative analysis of such attacks on image processing systems based on artificial intelligence. Approaches to the classification of attacks based on malicious perturbations are formulated. Defense methods against such attacks are considered, their shortcomings are revealed. The limitations of the applied defense methods that reduce the effectiveness of counteraction to attacks are shown. Approaches and practical measures to detect and eliminate harmful disturbances are proposed.

Keywords: artificial intelligence, artificial neural network, image processing, adversarial attack, backdoor embedding, adversarial perturbation, adversarial learning, defense distillation, feature squeezing, certified defense, data preprocessing

References
  1. Goldberg Y. A primer on neural network models for natural language processing. Journal of Artificial Intelligence Research, 2016, vol. 57, pp. 345–420. https://doi.org/10.1613/jair.4992
  2. Nassif A.B., Shahin I., Attili I., Azzeh M., Shaalan K. Speech recognition using deep neural networks: A systematic review. IEEE Access, 2019, vol. 7, pp. 19143–19165. https://doi.org/10.1109/access.2019.2896880
  3. Almabdy S., Elrefaei L. Deep convolutional neural network-based approaches for face recognition. Applied Sciences, 2019, vol. 9, no. 20, pp. 4397. https://doi.org/10.3390/app9204397
  4. Khan M.Z., Harous S., Hassan S. U., Khan M. U. G., Iqbal R., Mumtaz S. Deep unified model for face recognition based on convolution neural network and edge computing. IEEE Access, 2019, vol. 7, pp. 72622–72633. https://doi.org/10.1109/access.2019.2918275
  5. Zhang Y., Shi D., Zhan X., Cao D., Zhu K., Li Z. Slim-ResCNN: A deep residual convolutional neural network for fingerprint liveness detection. IEEE Access, 2019, vol. 7, pp. 91476–91487. https://doi.org/10.1109/access.2019.2927357
  6. Sarvamangala D.R., Kulkarni R.V. Convolutional neural networks in medical image understanding: a survey. Evolutionary Intelligence, 2022, vol. 15, no. 1, pp. 1–22. https://doi.org/10.1007/s12065-020-00540-3
  7. Mahmood M., Al-Khateeb B., Alwash W. A review on neural networks approach on classifying cancers. IAES International Journal of Artificial Intelligence, 2020, vol. 9, no. 2, pp. 317–326. http://doi.org/10.11591/ijai.v9.i2.pp317-326
  8. Singh V., Singh S., Gupta P. Real-time anomaly recognition through CCTV using neural networks. Procedia Computer Science, 2020, vol. 173, pp. 254–263. https://doi.org/10.1016/j.procs.2020.06.030
  9. Severino A., Curto S., Barberi S., Arena F., Pau G. Autonomous vehicles: an analysis both on their distinctiveness and the potential impact on urban transport systems. Applied Sciences, 2021,vol. 11, no. 8, pp. 3604. https://doi.org/10.3390/app11083604
  10. Wang L., Fan X., Chen J., Cheng J., Tan J., Ma X. 3D object detection based on sparse convolution neural network and feature fusion for autonomous driving in smart cities. Sustainable Cities and Society, 2020, vol. 54, pp. 102002. https://doi.org/10.1016/j.scs.2019.102002
  11. Chen L., Lin S., Lu X., Cao D., Wu H., Guo C., Wang F. Y. Deep neural network based vehicle and pedestrian detection for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, 2021, vol. 22, no. 6, pp. 3234–3246. https://doi.org/10.1109/tits.2020.2993926
  12. Chen P. Y., Liu S. Holistic adversarial robustness of deep learning models. Proceedings of the AAAI Conference on Artificial Intelligence, 2023, vol. 37, no. 13, pp. 15411–15420. https://doi.org/10.1609/aaai.v37i13.26797
  13. Huang X., Kroening D., Ruan W., Sharp J., Sun Y., Thamo E., Min W., Yi X. A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretability. Computer Science Review, 2020, vol. 37, pp. 100270. https://doi.org/10.1016/j.cosrev.2020.100270
  14. Szegedy C., Zaremba W., Sutskever I., Bruna J., Erhan D., Goodfellow I., Fergus R. Intriguing properties of neural networks. arXiv, 2013, arXiv:1312.6199. https://doi.org/10.48550/arXiv.1312.6199
  15. Song Y., Shu R., Kushman N., Ermon S. Constructing unrestricted adversarial examples with generative models. Advances in Neural Information Processing Systems, 2018, vol. 31.
  16. Sayghe A., Zhao J., Konstantinou C. Evasion attacks with adversarial deep learning against power system state estimation. Proc. of the 2020 IEEE Power & Energy Society General Meeting (PESGM), 2020, pp. 1–5. https://doi.org/10.1109/pesgm41954.2020.9281719
  17. Goodfellow I.J., Shlens J., Szegedy C. Explaining and harnessing adversarial examples. arXiv, 2014, arXiv:1412.6572. https://doi.org/10.48550/arXiv.1412.6572
  18. Paul R., Schabath M., Gillies R., Hall L., Goldgof D. Mitigating adversarial attacks on medical image understanding systems. Proc. of the 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), 2020, pp. 1517–1521. https://doi.org/10.1109/isbi45749.2020.9098740
  19. Rozsa A., Rudd E.M., Boult T.E. Adversarial diversity and hard positive generation. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2016, pp. 25–32. https://doi.org/10.1109/cvprw.2016.58
  20. Dong Y., Liao F., Pang T., Su H., Zhu J., Hu X., Li J. Boosting adversarial attacks with momentum. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 9185–9193. https://doi.org/10.1109/cvpr.2018.00957
  21. Miyato T., Maeda S.I., Koyama M., Ishii S. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, vol. 41, no. 8, pp. 1979–1993. https://doi.org/10.1109/tpami.2018.2858821
  22. Kurakin A., Goodfellow I.J., Bengio S. Adversarial examples in the physical world. Artificial Intelligence Safety and Security. Chapman and Hall/CRC, 2018, pp. 99–112. https://doi.org/10.1201/9781351251389-8
  23. Mądry A., Makelov A., Schmidt L., Tsipras D., Vladu A. Towards deep learning models resistant to adversarial attacks. Stat, 2017, vol. 1050, pp. 9.
  24. Xie C., Zhang Z., Zhou Y., Bai S., Wang J., Ren Z., Yuille A.L. Improving transferability of adversarial examples with input diversity. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 2730–2739. https://doi.org/10.1109/cvpr.2019.00284
  25. Dong X., Han J., Chen D., Liu J., Bian H., Ma Z., Li H., Wang X., Zhang W., Yu N. Robust superpixel-guided attentional adversarial attack. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 12895–12904. https://doi.org/10.1109/cvpr42600.2020.01291
  26. Sriramanan G., Addepalli S., Baburaj A. Guided adversarial attack for evaluating and enhancing adversarial defenses. Advances in Neural Information Processing Systems, 2020, vol. 33, pp. 20297–20308.
  27. Rony J., Hafemann L.G., Oliveira L.S., Ayed I.B., Sabourin R., Granger E. Decoupling direction and norm for efficient gradient-based L2 adversarial attacks and defenses. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 4322–4330. https://doi.org/10.1109/cvpr.2019.00445
  28. Moosavi-Dezfooli S.M., Fawzi A., Frossard P. DeepFool: a simple and accurate method to fool deep neural networks. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 2574–2582. https://doi.org/10.1109/cvpr.2016.282
  29. Carlini N., Wagner D. Towards evaluating the robustness of neural networks. Proc. of the IEEE Symposium on Security and Privacy (SP), 2017, pp. 39–57. https://doi.org/10.1109/sp.2017.49
  30. Yao Z., Gholami A., Xu P., Keutzer K., Mahoney M. W. Trust region based adversarial attack on neural networks. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 11350–11359. https://doi.org/10.1109/cvpr.2019.01161
  31. Papernot N., McDaniel P., Jha S., Fredrikson M., Celik Z. B., Swami A. The limitations of deep learning in adversarial settings. Proc. of the 2016 IEEE European Symposium on Security and Privacy (EuroS&P), 2016, pp. 372–387. https://doi.org/10.1109/eurosp.2016.36
  32. Su J., Vargas D.V., Sakurai K. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation, 2019, vol. 23, no. 5, pp. 828–841. https://doi.org/10.1109/tevc.2019.2890858
  33. Moosavi-Dezfooli S.M., Fawzi A., Fawzi O., Frossard P. Universal adversarial perturbations. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 1765–1773. https://doi.org/10.1109/cvpr.2017.17
  34. Brendel W., Rauber J., Bethge M. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. Advances in Reliably Evaluating and Improving Adversarial Robustness, 2021, pp. 77.
  35. Chen J., Jordan M.I., Wainwright M.J. HopSkipJumpAttack: A query-efficient decision-based attack.Proc. of the 2020 IEEE Symposium on Security and Privacy (SP), 2020, pp. 1277–1294. https://doi.org/10.1109/sp40000.2020.00045
  36. Liu Y., Moosavi-Dezfooli S.M., Frossard P. A geometry-inspired decision-based attack. Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 4890–4898. https://doi.org/10.1109/iccv.2019.00499
  37. Rahmati A., Moosavi-Dezfooli, S.M., Frossard P., Dai H. GeoDA: a geometric framework for black-box adversarial attacks. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 8446–8455. https://doi.org/10.1109/cvpr42600.2020.00847
  38. Du J., Zhang H., Zhou J.T., Yang Y., Feng J. Query-efficient meta attack to deep neural networks. Proc. of the International Conference on Learning Representations, 2020.
  39. Li J., Ji R., Liu H., Liu J., Zhong B., Deng C., Tian Q. Projection & probability-driven black-box attack. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 362–371. https://doi.org/10.1109/cvpr42600.2020.00044
  40. Li H., Xu X., Zhang X., Yang S., Li B. QEBA: Query-efficient boundary-based blackbox attack. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 1221–1230. https://doi.org/10.1109/cvpr42600.2020.00130
  41. Cheng M., Singh S., Chen P., Chen P.Y., Liu S., Hsieh C.J. Sign-OPT: A query-efficient hard-label adversarial attack. Proc. of the International Conference on Learning Representations, 2020.
  42. Brunner T., Diehl F., Le M.T., Knoll A. Guessing smart: Biased sampling for efficient black-box adversarial attacks. Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 4958–4966. https://doi.org/10.1109/iccv.2019.00506
  43. Maho T., Furon T., Le Merrer E. SurFree: a fast surrogate-free black-box attack. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 10430–10439. https://doi.org/10.1109/cvpr46437.2021.01029
  44. Shi Y., Han Y., Tian Q. Polishing decision-based adversarial noise with a customized sampling. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 1030–1038. https://doi.org/10.1109/cvpr42600.2020.00111
  45. Huang Z., Zhang T. Black-box adversarial attack with transferable model-based embedding. Proc. of the International Conference on Learning Representations, 2020.
  46. Zhou M., Wu J., Liu Y., Liu S., Zhu C. DaST: Data-free substitute training for adversarial attacks. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 234–243. https://doi.org/10.1109/cvpr42600.2020.00031
  47. Zou J., Pan Z., Qiu J., Liu X., Rui T., Li W. Improving the transferability of adversarial examples with resized-diverse-inputs, diversity-ensemble and region fitting. Lecture Notes in Computer Science, 2020, vol. 12367, pp. 563–579. https://doi.org/10.1007/978-3-030-58542-6_34
  48. Wang X., He K. Enhancing the transferability of adversarial attacks through variance tuning. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 1924–1933. https://doi.org/10.1109/cvpr46437.2021.00196
  49. Wu W., Su Y., Lyu M.R., King I. Improving the transferability of adversarial samples with adversarial transformations. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 9024–9033. https://doi.org/10.1109/cvpr46437.2021.00891
  50. Hosseini H., Poovendran R. Semantic adversarial examples. Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2018, pp. 1614–1619. https://doi.org/10.1109/cvprw.2018.00212
  51. Engstrom L., Tran B., Tsipras D., Schmidt L., Madry A. A rotation and a translation suffice: Fooling cnns with simple transformations. Available at: https://openreview.net/forum?id=BJfvknCqFQ (accessed: 29.05.2023).
  52. Joshi A., Mukherjee A., Sarkar S., Hegde C. Semantic adversarial attacks: Parametric transformations that fool deep classifiers. Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 4773–4783. https://doi.org/10.1109/iccv.2019.00487
  53. Liu A., Wang J., Liu X., Cao B., Zhang C., Yu H. Bias-based universal adversarial patch attack for automatic check-out. Lecture Notes in Computer Science, 2020, vol. 12358, pp. 395–410. https://doi.org/10.1007/978-3-030-58601-0_24
  54. Swathi P., Sk S. DeepFake creation and detection: A survey. Proc. of the 2021 Third International Conference on Inventive Research in Computing Applications (ICIRCA), 2021, pp. 584–588. https://doi.org/10.1109/icirca51532.2021.9544522
  55. Chadha A., Kumar V., Kashyap S., Gupta M. Deepfake: An Overview. Lecture Notes in Networks and Systems, 2021, vol. 203, pp. 557–566. https://doi.org/10.1007/978-981-16-0733-2_39
  56. Nakka K.K., Salzmann M. Indirect local attacks for context-aware semantic segmentation networks. Lecture Notes in Computer Science, 2020, vol. 12350, pp. 611–628. https://doi.org/10.1007/978-3-030-58558-7_36
  57. He Y., Rahimian S., Schiele B., Fritz M. Segmentations-leak: Membership inference attacks and defenses in semantic image segmentation. Lecture Notes in Computer Science, 2020, vol. 12368, pp. 519–535. https://doi.org/10.1007/978-3-030-58592-1_31
  58. Choi J.H. Zhang H., Kim J.H., Hsieh C.J., Lee J.S. Evaluating robustness of deep image super-resolution against adversarial attacks. Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 303–311. https://doi.org/10.1109/iccv.2019.00039
  59. Jiang L., Ma X., Chen S., Bailey J., Jiang Y.G. Black-box adversarial attacks on video recognition models. Proc. of the 27th ACM International Conference on Multimedia, 2019, pp. 864–872. https://doi.org/10.1145/3343031.3351088
  60. Li S., Aich A., Zhu S., Asif S., Song C., Roy-Chowdhury A., Krishnamurthy S. Adversarial attacks on black box video classifiers: Leveraging the power of geometric transformations. Advances in Neural Information Processing Systems, 2021, vol. 34, pp. 2085–2096.
  61. Chen X., Li S., Huang H. Adversarial attack and defense on deep neural network-based voice processing systems: An overview. Applied Sciences, 2021, vol. 11, no. 18, pp. 8450. https://doi.org/10.3390/app11188450
  62. Kwon H., Kim Y., Yoon H., Choi D. Selective audio adversarial example in evasion attack on speech recognition system. IEEE Transactions on Information Forensics and Security, 2020, vol. 15, pp. 526–538. https://doi.org/10.1109/tifs.2019.2925452
  63. Usama M., Qayyum A., Qadir J., Al-Fuqaha A. Black-box adversarial machine learning attack on network traffic classification. Proc. of the 15th International Wireless Communications & Mobile Computing Conference (IWCMC), 2019, pp. 84–89. https://doi.org/10.1109/iwcmc.2019.8766505
  64. Imam N.H., Vassilakis V.G. A survey of attacks against twitter spam detectors in an adversarial environment. Robotics, 2019, vol. 8, no. 3, pp. 50. https://doi.org/10.3390/robotics8030050
  65. Zhong H., Liao C., Squicciarini A.C., Zhu S., Miller D. Backdoor embedding in convolutional neural network models via invisible perturbation. Proc. of the Tenth ACM Conference on Data and Application Security and Privacy, 2020, pp. 97–108. https://doi.org/10.1145/3374664.3375751
  66. Liu X., Yang H., Liu Z., Song L., Li H., Chen Y. Dpatch: An adversarial patch attack on object detectors. arXiv, 2018, arXiv:1806.02299. https://doi.org/10.48550/arXiv.1806.02299
  67. Liu Y., Ma X., Bailey J., Lu F. Reflection backdoor: A natural backdoor attack on deep neural networks. Lecture Notes in Computer Science, 2020, vol. 12355, pp. 182–199. https://doi.org/10.1007/978-3-030-58607-2_11
  68. Nguyen A., Tran A. WaNet - imperceptible warping-based backdoor attack. Proc. of the International Conference on Learning Representations, 2021.
  69. Kostyumov V. A Survey and systematization of evasion attacks in computer vision. International Journal of Open Information Technologies, 2022, vol. 10, no. 10, pp. 11–20. (in Russian)
  70. Papernot N., McDaniel P., Wu X., Jha S., Swami A. Distillation as a defense to adversarial perturbations against deep neural networks. Proc. of the 2016 IEEE Symposium on Security and Privacy (SP), 2016, pp. 582–597. https://doi.org/10.1109/sp.2016.41
  71. Steihaug T. The conjugate gradient method and trust regions in large scale optimization. SIAM Journal on Numerical Analysis, 1983, vol. 20, no. 3, pp. 626–637. https://doi.org/10.1137/0720042
  72. Curtis A.R., Powell M.J.D., Reid J.K. On the estimation of sparse Jacobian matrices. IMA Journal of Applied Mathematics, 1974, vol. 13, no. 1, pp. 117–120. https://doi.org/10.1093/imamat/13.1.117
  73. Niebur E. Saliency map. Scholarpedia, 2007, vol. 2, no. 8, pp. 2675. https://doi.org/10.4249/scholarpedia.2675
  74. Das S., Suganthan P.N. Differential evolution: A survey of the state-of-the-art. IEEE Transactions on Evolutionary Computation, 2011, vol. 15, no. 1, pp. 4–31. https://doi.org/10.1109/tevc.2010.2059031
  75. Badrinarayanan V., Kendall A., Cipolla R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, vol. 39, no. 12, pp. 2481–2495. https://doi.org/10.1109/tpami.2016.2644615
  76. Lowd D., Meek C. Adversarial learning. Proc. of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, 2005, pp. 641–647. https://doi.org/10.1145/1081870.1081950
  77. Xu W., Evans D., Qi Y. Feature squeezing: Detecting adversarial examples in deep neural networks. Proc. of the 2018 Network and Distributed System Security Symposium (NDSS), 2018.
  78. Liao F., Liang M., Dong Y., Pang T., Hu X., Zhu J. Defense against adversarial attacks using high-level representation guided denoiser. Proc. of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 1778–1787. https://doi.org/10.1109/cvpr.2018.00191
  79. Zhang D., Ye M., Gong C., Zhu Z., Liu Q. Black-box certification with randomized smoothing: A functional optimization based framework. Advances in Neural Information Processing Systems, 2020, vol. 33, pp. 2316–2326.
  80. Fischer M., Baader M., Vechev M. Certified defense to image transformations via randomized smoothing. Advances in Neural Information Processing Systems, 2020, vol. 33, pp. 8404–8417.
  81. Yang R., Chen X.Q., Cao T.J. APE-GAN++: An improved APE-GAN to eliminate adversarial perturbations. IAENG International Journal of Computer Science, 2021, vol. 48, no. 3, pp. 827–844.
  82. Glenn T.C., Zare A., Gader P.D. Bayesian fuzzy clustering. IEEE Transactions on Fuzzy Systems, 2015, vol. 23, no. 5, pp. 1545–1561. https://doi.org/10.1109/tfuzz.2014.2370676
  83. Plackett R.L. Karl Pearson and the chi-squared test. International Statistical Review / Revue Internationale de Statistique, 1983, vol. 51, no. 1, pp. 59–72. https://doi.org/10.2307/1402731
  84. McLachlan G.J. Mahalanobis distance. Resonance, 1999, vol. 4, no. 6, pp. 20–26. https://doi.org/10.1007/BF02834632


Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
Copyright 2001-2024 ©
Scientific and Technical Journal
of Information Technologies, Mechanics and Optics.
All rights reserved.

Яндекс.Метрика