
Nikiforov
Vladimir O.
D.Sc., Prof.
doi: 10.17586/2226-1494-2025-25-4-744-754
Algorithm for human interaction with a model of an industrial cyber-physical system by means of neural interface
Read the full article

For citation:
Sizov M.S., Marusina M.Y., Kipriianov K.V., Arckhipov V.A., Lou J., Nagornova Zh.V., Shemyakina N.V. Algorithm for human interaction with a model of an industrial cyber-physical system by means of neural interface. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2025, vol. 25, no. 4, pp. 744–754 (in Russian). doi: 10.17586/2226-1494-2025-25-4-744-754
Abstract
The article proposes an algorithm of a Brain Computer Interface (BCI) for implementation of interaction between a human and a model of an industrial cyberphysical system. The interface facilitates selecting a conceived tool on the basis of the classification of evoked responses of a test person’s encephalogram to visual stimuli (tool images). To conduct the study there has been designed a software system operated with a web-server, a controller, and a user BCI. The cerebral bioelectrical activity of a test person has been constantly registered with the encephalograph produced by LLC MITSAR followed by online signal processing conducted by the designed original software system. The stored evoked responses to stimuli have been classified in a variety of ways — peak-based selection, a support vector machine, and a neural net. There has been proved that accuracy of the classification of evoked potentials both with the help of a neural net and a support vector machine are approximately equal and these algorithms can be implemented in the online mode. Analysis of the experiments performed has shown that the proposed algorithm makes it possible to classify presented visual stimuli in neural interfaces in the online mode. The results show how it is possible to organize a ‘deeply integrated’ interaction between a human and an equipment through an impact of commands based on the processed signals of bioelectrical brain activity of a human on a 3D model of a production site.
References
- Nazarenko A.A., Camarinha-Matos L.M. A human-al framework to design collaborative cyber physical systems. IFIP Advances in Information and Communication Technology, 2024, vol. 716, pp. 28–42. https://doi.org/10.1007/978-3-031-63851-0_2
- Colombo A.W., Karnouskos S., Hanisch C. Engineering human-focused Industrial Cyber-Physical Systems in Industry 4.0 context. Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences, 2021, vol. 379, no. 2207, pp. 20200366. https://doi.org/10.1098/rsta.2020.0366
- Schmidt D., Diez J.V., Ordieres-Meré J., Gevers R., Schwiep J., Molina M. Industry 4.0 lean shopfloor management characterization using EEG sensors and deep learning. Sensors, 2020, vol. 20, no. 10, pp. 2860. https://doi.org/10.3390/s20102860
- Piardi L., Leitão P., Queiroz J., Pontes J. Role of digital technologies to enhance the human integration in industrial cyber–physical systems. Annual Reviews in Control, 2024, vol. 57, pp. 100934. https://doi.org/10.1016/j.arcontrol.2024.100934
- Zhou J., Zhou Y., Wang B., Zang J. Human–Cyber–Physical Systems (HCPSs) in the context of new-generation intelligent manufacturing. Engineering, 2019, vol. 5, no. 4, pp. 624–636. https://doi.org/10.1016/j.eng.2019.07.015
- Slavic D., Marjanovic U., Pezzotta G., Turcin I., Rakic S. Servitization and industry 5.0: the future trends of manufacturing transformation. IFIP Advances in Information and Communication Technology, 2023, vol. 690, pp. 109–121. https://doi.org/10.1007/978-3-031-43666-6_8
- Slavic D., Romero D., Pezzotta G., Marjanovic U., Savkovic B., Popan I.A., Rakic S. Towards human-centric digital services: a development framework. IFIP Advances in Information and Communication Technology, 2024, vol. 732, pp. 184–197. https://doi.org/10.1007/978-3-031-71637-9_13
- Reddy K.K., Reddy A.V., Doss S., Priyanka K. Human-machine collaboration and emotional intelligence in Industry 6.0: concepts, challenges, and future directions. Examining the Metaverse in Healthcare, 2024, pp. 221–246. https://doi.org/10.4018/979-8-3693-1515-6.ch009
- Babkin A.V., Shkarupeta E.V. Industry 6.0: the essence, trends and strategic opportunities for Russia. Russian Journal of Industrial Economics, 2024, vol.17, no. 4, pp. 353-377. (in Russian). https://doi.org/10.17073/2072-1633-2024-4-1369
- Värbu K., Muhammad N., Muhammad Y. Past, present, and future of EEG-based BCI applications. Sensors, 2022, vol. 2, no. 9, pp. 3331. https://doi.org/10.3390/s22093331
- Kitchener P.D., Hales C.G. What neuroscientists think, and don’t think, about consciousness. Frontiers in Human Neuroscience, 2022, vol. 16, pp. 767612. https://doi.org/10.3389/fnhum.2022.767612
- Douibi K., Le Bars S., Lemontey A., Nag L., Balp R., Breda G. Toward EEG-based BCI applications for Industry 4.0: challenges and possible applications. Frontiers in Human Neuroscience, 2021, vol. 15, pp. 705064. https://doi.org/10.3389/fnhum.2021.705064
- Gundelakh F., Stankevich L., Sonkin K., Nagornova G., Shemyakina N. Application of Brain-computer interfaces in assistive technologies. SPIIRAS Proceedings, vol. 19, no. 2, pp. 277–301. (in Russian). https://doi.org/10.15622/sp.2020.19.2.2
- Gundelakh F., Stankevich L., Sonkin K. Mobile robot control based on noninvasive brain-computer interface using hierarchical classifier of imagined motor commands. MATEC Web of Conferences, 2018, vol. 161, pp. 03003. https://doi.org/10.1051/matecconf/201816103003
- Kapralov N., Nagornova Z., Shemyakina N. Classification methods for EEG patterns of imaginary movements. Informatics and Automation, 2021. vol. 20, no. 1, pp. 94–132. (in Russian). https://doi.org/10.15622/ia.2021.20.1.4
- Guo Z., Wu J., Song Y., Bu J., Mai W., Zheng Q., Ouyang W., Song C. Neuro-3D: towards 3D visual decoding from EEG signals. arXiv, 2024, arXiv:2411.12248. https://doi.org/10.48550/arXiv.2411.12248
- Uchimura T. Robots or 3D models control by Brain-Computer Interfaces. Dissertation for the degree of Master of Science (Computer Science). Flinders University, Adelaide, 2021. 40 p.
- Cao B., Niu H., Hao J., Wang G. Building EEG-based CAD object selection intention discrimination model using convolutional neural network (CNN). Advanced Engineering Informatics, 2022, vol. 52, pp. 101548. https://doi.org/10.1016/j.aei.2022.101548
- Duncana C.C., Barryb R.J., Connollyc J.F., Fischer C., Michie P.T., Näätänen R., et al. Event-related potentials in clinical research: Guidelines for eliciting, recording, and quantifying mismatch negativity, P300, and N400. Clinical Neurophysiology, 2009, vol. 120, no. 11, pp. 1883–1908. https://doi.org/10.1016/j.clinph.2009.07.045
- Gonzalez-Santocildes A., Vazquez J.-I., Eguiluz A. Enhancing robot behavior with EEG, reinforcement learning and beyond: a review of techniques in collaborative robotics. Applied Sciences, 2024, vol. 14, no. 14, pp. 6345. https://doi.org/10.3390/app14146345
- Aleksandrov M.V., Ivanov L.B., Lytaev S.A. Electroencephalography. St. Petersburg, SpecLit Publ.,2020, 224 p. (in Russian)
- Marusina M.Ya., Burdaev I.V. Automatic recognition of visual stimules by single evoked potentials on an electroencephalogram. Journal of Instrument Engineering, 2023, vol. 66, no. 3, pp. 247-250. (in Russian). https://doi.org/10.17586/0021-3454-2023-66-3-247-250
- Baiano C., Zeppieri M. Visual Evoked Potential. StatPearls [Internet], 2023, PMID: 35881733.
- Flach P. Machine Learning: The Art and Science of Algorithms that Make Sense of Data. Cambridge University Press, 2012, 409 p.
- Weidman S. Deep Learning from Scratch: Building with Python from First Principles. O'Reilly Media, 2019, 250 p.
- Qi J., Hu J., Peng Y., Ren Q., Wang W., Zhan Z. Integration of similarity measurement and dynamic SVM for electrically evoked potentials prediction in visual prostheses research. Expert Systems with Applications, 2011, vol. 38, no. 5, pp. 5044–5060. https://doi.org/10.1016/j.eswa.2010.09.145
- Swihart S.L., Matheny A. Classification of chromatic visual evoked potentials with the aid of a neural net. Computers in Biology and Medicine, 1992, vol. 22, no. 3, pp. 165–171. https://doi.org/10.1016/0010-4825(92)90012-c
- Zheng X., Cao Z., Bai Q. An evoked potential-guided deep learning brain representation for visual classification. Communications in Computer and Information Science, 2020, vol. 1333, pp. 54–61. https://doi.org/10.1007/978-3-030-63823-8_7
- Zhang Y., Xie S., Wang H., Zhang Z. Data analytics in steady-state visual evoked potential-based brain-computer interface: a review. IEEE Sensors Journal, 2021, vol. 21, no. 2, pp. 1124–1138. https://doi.org/10.1109/jsen.2020.3017491
- Cui Z., Chen W., Chen Y. Multi-scale convolutional neural networks for time series classification. arXiv, 2016, arXiv:1603.06995. https://arxiv.org/abs/1603.06995