Menu
Publications
2026
2025
2024
2023
2022
2021
2020
2019
2018
2017
2016
2015
2014
2013
2012
2011
2010
2009
2008
2007
2006
2005
2004
2003
2002
2001
Editor-in-Chief
Nikiforov
Vladimir O.
D.Sc., Prof.
Partners
doi: 10.17586/2226-1494-2026-26-5-1000-1008
Training visual language models for MRI-based diagnostics
Read the full article
Article in Russian
For citation:
Abstract
For citation:
Gusarova N.F., Dobrenko N.V., Zubanenko A.A., Tomilov I.V., Ulyanov P.G. Training visual language models for MRI-based diagnostics. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2026, vol. 26, no. 5, pp. 1000–1008 (in Russian). doi: 10.17586/2226-1494-2026-26-5-1000-1008
Abstract
In modern radiology, diagnostic Artificial Intelligence assistants for classifying Magnetic Resonance Imaging (MRI) brain images are typically based on Convolutional Neural Networks (CNNs), which work exclusively with the visual signal and are unable to use structured diagnostic descriptions of pathologies. Vision Language Models (VLMs) open up new possibilities: joint processing of visual and textual data allows for the consideration of the semantic context of pathologies, which is inaccessible to CNN classifiers. The key obstacle to their application remains the high resource intensity of retraining, which is incomparable with the capabilities of a typical clinic. This paper proposes a method for constructing a resource-efficient diagnostic AI assistant based on VLM Gemma3 (4b-it) using low-rank adaptation (LoRA). Retraining was performed on a dataset of 91,777 Fluid-Attenuated Inversion Recovery MRI brain images, distributed across eight pathology classes. Only low-rank adapters were trained — 12.5 % of the model parameters, which allowed training on a single NVIDIA RTX3090 Graphics Processing Unit with 24 GB of Video Random Access Memory without masked data annotation. The radiologist’s expert knowledge was integrated via a system prompt, which assigns the model the role of an experienced neuroradiologist with explicit analysis criteria. On the validation set (9178 examples), the model achieved: Accuracy 0.930, F1-score 0.929, Specificity 0.989, and Cohen’s Kappa 0.915. Comparison with the baseline ConvNeXt-Base classifier on an identical dataset showed the superiority of VLM+LoRA across all metrics: Accuracy increased by 7.4 percentage points, and Cohen’s Kappa by 0.090. The obtained results confirm the practical applicability of the proposed approach for clinics with limited computing resources.
Keywords: multimodal language model, diagnostic AI assistant, brain MRI, FLAIR sequence, low-rank adaptation, multi-class classification, brain pathologies, resource-efficient learning, Gemma, LoRA
Acknowledgements. This work was supported by the Ministry of Science and Higher Education of the Russian Federation, Goszadanie (State Assignment) FSER-2025-0013.
Acknowledgements. This work was supported by the Ministry of Science and Higher Education of the Russian Federation, Goszadanie (State Assignment) FSER-2025-0013.

