Position Papers of the 21st Conference on Computer Science and Intelligence Systems

DOI: http://dx.doi.org/10.15439/2026F1450

Citation: Saulo Silva (2026). Regularizing the Prompt: Stable Model-Agnostic LLM Responses Through Inversion Theory. In M. Bolanowski, M. Ganzha, M. Grzegorowski, L. Maciaszek, M. Paprzycki, A. Paszkiewicz, D. Ślęzak (eds), Proceedings of the 21st Conference on Computer Science and Intelligence Systems. ACSIS, Vol. 48, pages 157–164.

Full text

Abstract. Large language models from different vendors, tiers and versions exhibit divergent behavioral characteristics when given identical prompts and input, making model-agnostic solutions a non-trivial engineering problem. This article analyzes the statistical foundations of modern models, identifies stochastic base training and vendor-specific alignment fine-tuning as two sources of divergence, and approaches prompt engineering as an ill-posed inverse problem, using geophysical inversion as the mathematical foundation. The prompt is formalized as a regularization instrument analogous to Tikhonov regularization, where insufficient or excessive constraints respectively produce unstable or oversmoothed responses. A methodology for finding the minimum set of constraints that all target models consistently follow is presented and validated through Summerizer, an iOS application that interfaces with multiple models across vendors without prompt branching, providing a foundation for future platform-independent multi-model implementations.

References

1. L. W. B. Leite, Conceitos da Análise Espectral de Sinais em Geofı́sica, 1st ed. UFPA-IG, 2015.
2. A. N. Tikhonov and V. Y. Arsenin, Solutions of Ill-Posed Problems. Winston & Sons, 1977.
3. W. Menke, Geophysical Data Analysis: Discrete Inverse Theory, 2nd ed. Academic Press, 1989.
4. A. Tarantola, Inverse Problem Theory and Methods for Model Parameter Estimation. Society for Industrial and Applied Mathematics, 2005.
5. I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016.
6. K. Aki and P. G. Richards, Quantitative Seismology, 2nd ed. University Science Books, 2002.
7. C. M. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
8. R. M. Gray, Entropy and Information Theory. Berlin/Heidelberg, Germany: Springer Science & Business Media, 2011.
9. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, pp. 533–536, 1986. https://dx.doi.org/10.1038/323533a0
10. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems, vol. 30, 2017, https://arxiv.org/abs/1706.03762.
11. D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations, 2015, arXiv:1412.6980.
12. L. Ouyang et al., “Training language models to follow instructions with human feedback,” in Advances in Neural Information Processing Systems, vol. 35, 2022, arXiv:2203.02155.
13. D. Sculley, G. Holt, D. Golovin, E. Davydov, J.-F. Crespo, D. Dennison, T. Phillips, D. Ebner, V. Chaudhary, and M. Young, “Hidden technical debt in machine learning systems,” in Advances in Neural Information Processing Systems, vol. 28, 2015.
14. T. Shin, Y. Razeghi, R. L. Logan IV, E. Wallace, and S. Singh, “AutoPrompt: Eliciting knowledge from language models with automatically generated prompts,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020. https://dx.doi.org/10.18653/v1/2020.emnlp-main.346 pp. 4222–4235.
15. B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021. https://dx.doi.org/10.18653/v1/2021.emnlp-main.243 pp. 3045–3059.