doi: 10.17586/2226-1494-2026-26-5-1000-1008


Training visual language models for MRI-based diagnostics

N. F. Gusarova, N. V. Dobrenko, A. A. Zubanenko, I. V. Tomilov, P. G. Ulyanov


Read the full article  ';
Article in Russian

For citation:
Gusarova N.F., Dobrenko N.V., Zubanenko A.A., Tomilov I.V., Ulyanov P.G. Training visual language models for MRI-based diagnostics. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2026, vol. 26, no. 5, pp. 1000–1008 (in Russian). doi: 10.17586/2226-1494-2026-26-5-1000-1008


Abstract
In modern radiology, diagnostic Artificial Intelligence assistants for classifying Magnetic Resonance Imaging (MRI) brain images are typically based on Convolutional Neural Networks (CNNs), which work exclusively with the visual signal and are unable to use structured diagnostic descriptions of pathologies. Vision Language Models (VLMs) open up new possibilities: joint processing of visual and textual data allows for the consideration of the semantic context of pathologies, which is inaccessible to CNN classifiers. The key obstacle to their application remains the high resource intensity of retraining, which is incomparable with the capabilities of a typical clinic. This paper proposes a method for constructing a resource-efficient diagnostic AI assistant based on VLM Gemma3 (4b-it) using low-rank adaptation (LoRA). Retraining was performed on a dataset of 91,777 Fluid-Attenuated Inversion Recovery MRI brain images, distributed across eight pathology classes. Only low-rank adapters were trained — 12.5 % of the model parameters, which allowed training on a single NVIDIA RTX3090 Graphics Processing Unit with 24 GB of Video Random Access Memory without masked data annotation. The radiologist’s expert knowledge was integrated via a system prompt, which assigns the model the role of an experienced neuroradiologist with explicit analysis criteria. On the validation set (9178 examples), the model achieved: Accuracy 0.930, F1-score 0.929, Specificity 0.989, and Cohen’s Kappa 0.915. Comparison with the baseline ConvNeXt-Base classifier on an identical dataset showed the superiority of VLM+LoRA across all metrics: Accuracy increased by 7.4 percentage points, and Cohen’s Kappa by 0.090. The obtained results confirm the practical applicability of the proposed approach for clinics with limited computing resources.

Keywords: multimodal language model, diagnostic AI assistant, brain MRI, FLAIR sequence, low-rank adaptation, multi-class classification, brain pathologies, resource-efficient learning, Gemma, LoRA

Acknowledgements. This work was supported by the Ministry of Science and Higher Education of the Russian Federation, Goszadanie (State Assignment) FSER-2025-0013.

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
Copyright 2001-2026 ©
Scientific and Technical Journal
of Information Technologies, Mechanics and Optics.

Яндекс.Метрика