Toward Conversational Decision Support Systems: Integrating LLMs in the Operations Research Methodology
Mariusz Kaleta
DOI: http://dx.doi.org/10.15439/2025F5420
Citation: Proceedings of the 20th Conference on Computer Science and Intelligence Systems (FedCSIS), M. Bolanowski, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 43, pages 165–174 (2025)
Abstract. This paper introduces the concept of Conversational Decision Support Systems (C-DSS)---a novel, agent-based framework that leverages Large Language Models (LLMs) to enhance the Operations Research (OR) methodology. We focus on the modeling and coding stages of decision support systems, where language-based interaction is crucial. The paper evaluates the effectiveness of LLMs in generating mathematical models and AMPL code for a curated set of 20 LP/MILP artifacts. Four architectural setups are analyzed: a monolithic LLM agent (M/C), its enhancement with a code verifier (M/C+V), agent-based decomposition with RAG-enhanced coding (M+C$^R$+V), and full specialization with RAG-enhanced modeling and coding (M$^R$+C$^R$+V). Experimental results on two benchmark problems reveal that the targeted retrieval-augmented generation technique (RAG) significantly improves performance for complex modeling patterns such as piecewise functions, indicator constraints, and nested logic. We also propose a broader vision of C-DSS as a multi-agent ecosystem---including agents for visualization, explanation, verification, and orchestration---suggesting a path toward more explainable, adaptable, and intelligent decision support systems.