Menu
Publications
2026
2025
2024
2023
2022
2021
2020
2019
2018
2017
2016
2015
2014
2013
2012
2011
2010
2009
2008
2007
2006
2005
2004
2003
2002
2001
Editor-in-Chief
Nikiforov
Vladimir O.
D.Sc., Prof.
Partners
doi: 10.17586/2226-1494-2026-26-4-844-850
Automatic prompt optimization with prompt distillation
Read the full article
';
Article in English
For citation:
Abstract
For citation:
Zhuravlev V.N., Khairullin A.R., Dyagin E.A., Sitkina A.N., Kulin N.I. Automatic prompt optimization with prompt distillation. Scientific and Technical Journal of Information Technologies, Mechanics and Optics, 2026, vol. 26, no. 4, pp. 844–850. doi: 10.17586/2226-1494-2026-26-4-844-850
Abstract
Autoprompting is the process of automatically selecting optimized prompts for language models which is gaining popularity due to the rapid development of prompt engineering driven by extensive research in the field of Large Language Models. This paper presents DistillPrompt — a novel autoprompting method based on Large Language Models that employs a multi-stage integration of task-specific information into prompts using training data. DistillPrompt utilizes distillation, compression, and aggregation operations to explore the prompt space more thoroughly. The method was tested on different datasets for text classification and generation tasks using t-lite-instruct-2.1, gpt-3.5-turbo and gpt-4o-mini language models. The results demonstrate a significant average improvement in key metrics over existing methods in the field, establishing DistillPrompt as one of the most effective non-gradient approaches in autoprompting.
Keywords: LLM, autoprompting, prompt distillation, prompting, prompt engineering
References
References
1. Kadavath S., Conerly T., Askell A., Henighan T., Drain D., Perez E., et al. Language models (mostly) know what they know. arXiv, 2022. arXiv:2207.05221. doi: 10.48550/arXiv.2207.05221
2. Wei J., Bosma M., Zhao V.Y., Guu K., Yu A.W., Lester B., et al. Finetuned language models are zero-shot learners. arXiv, 2021. arXiv:2109.01652. doi: 10.48550/arXiv.2109.01652
3. Brown T.B., Mann B., Ryder N., Subbiah M., Kaplan J., Dhariwal P., Neelakantan A., et al. Language models are few-shot learners.arXiv, 2020. arXiv:2005.14165. doi: 10.48550/arXiv.2005.14165
4. Wei J., Wang X., Schuurmans D., Bosma M., Ichter B., Xia F., et al. Chain-of-thought prompting elicits reasoning in large language models. Proc. of the 36th International Conference on Neural Information Processing Systems, 2022, pp. 24824–24837.
5. Li Z., Peng B., He P., Galley M., Gao J., Yan X. Guiding large language models via directional stimulus prompting. Proc. of the 37th International Conference on Neural Information Processing Systems, 2023, pp. 62630–62656.
6. Liu R., Geng J.,WuA.J., Sucholutsky I., Lombrozo T., Griffiths T.L. Mind your step (by step): chain-of-thought can reduce performance on tasks where thinking makes humans worse. Proc. of the 42nd International Conference on Machine Learning, 2025, pp. 38489–38517.
7. Guo D., Yang D., Zhang H., Song J., Wang P., Zhu Q., et al. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning.Nature, 2025, vol. 645, no. 8081, pp. 633–638. doi: 10.1038/s41586-025-09422-z
8. Zhou Y., Muresanu A.I., Han Z., Paster K., Pitis S., Chan H., et al. Large language models are human-level prompt engineers. Proc. of the 11th International Conference on Learning Representations, 2023.
9. Shin T., Razeghi Y., Logan R.L., Wallace E., Singh S. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. Proc. of the Conference on Empirical Methods in Natural Language Processing Proceedings of the Conference (EMNLP-2020), 2020, pp. 4222–4235.
10. Prasad A., Hasa P., Zhou X., Bansal M. Grips: Gradient-free, edit-based instruction search for prompting large language models.Proc. of the 17th Conference of the European Chapter of the Association for Computational Linguistics, 2023, pp. 3845–3864. doi: 10.18653/v1/2023.eacl-main.277
11. Pryzant R., Iter D., Li J., Lee Y., Zhu C., Zeng M. Automatic prompt optimization with "gradient descent" and beam search.Proc. of the Conference on Empirical Methods in Natural Language Processing, 2023, pp. 7957–7968. doi: 10.18653/v1/2023.emnlp-main.494
12. Li L., Zhang Y., Chen L. Prompt distillation for efficient llm-based recommendation. Proc. of the 32nd ACM International Conference on Information and Knowledge Management, 2023, pp. 1348–1357. doi: 10.1145/3583780.3615017
13. Yao S., Yu D., Zhao J., Shafran I., Griffiths T., Cao Y., et al. Tree of thoughts: Deliberate problem solving with large language models. Proc. of the 37th International Conference on Neural Information Processing Systems, 2023, pp. 11809–11822
