Logo PTI Logo FedCSIS

Proceedings of the 21st Conference on Computer Science and Intelligence Systems (FedCSIS)

Annals of Computer Science and Information Systems, Volume 47

Towards LLM-Based Context Engineering for Model-Driven Vibe-DevOps-Engineering of Cross-Platform CI/CD Pipeline Support

DOI: http://dx.doi.org/10.15439/2026F5402

Citation: Uldis Karlovs-Karlovskis (). Towards LLM-Based Context Engineering for Model-Driven Vibe-DevOps-Engineering of Cross-Platform CI/CD Pipeline Support. In M. Bolanowski, M. Ganzha, M. Grzegorowski, L. Maciaszek, M. Paprzycki, A. Paszkiewicz, D. Ślęzak (eds), Proceedings of the 21st Conference on Computer Science and Intelligence Systems (FedCSIS). ACSIS, Vol. 47, pages 83–93.

Full text

Abstract. Existing model-driven DevOps automation either depends on handcrafted transformations that do not scale to new CI/CD platforms or on prompt-based LLM generation that lacks determinism and reliability for production. This study introduces a proof of concept for vibe DevOps engineering, where LLM-based context engineering synthesizes model-driven transformation chains for CI/CD pipelines from existing MDE artifacts and platform documentation. Metamodels, ATL rules, Acceleo templates, and CI/CD specifications are decomposed and organized to prepare the ground for a future Graph-RAG and multi-agent architecture that will derive platform-specific DevOps artifacts from a platform-independent DevOps metamodel. In the current phase, the roles of the envisioned agents are emulated by a human researcher executing the chain of M2M2M2C steps and Eclipse-based runtime testing. Cross-platform constraint transfer reduces total refinement rounds from 11 (GitLab CI/CD) to 6 (Atlassian Bamboo) to 5 (Azure DevOps) across successive runs, and assess syntactic correctness, semantic alignment, and executability, indicating potential and current limitations of vibe DevOps engineering.

References

  1. J. R. Blazevics, O. Nikiforova, and O. Pastor, “A Framework for Model-Driven AI-Assisted Generation of IT Project Management Plan and Scope Documents,” Annals of Computer Science and Intelligence Systems, no. 2025, pp. 121–132, 2025. https://dx.doi.org/10.15439/2025F8736
  2. J. Humble and D. Farley, Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation, ser. A Martin Fowler Signature Book. Addison-Wesley, 2010. ISBN 9780321601919. [Online]. Available: https://books.google.lv/books?id= 9CAxmQEACAAJ
  3. O. Bedrina, “The State of CI/CD in 2025: Key Insights from the Latest JetBrains Survey,” JetBrains TeamCity Blog, 10 2025. [Online]. Available: https://blog.jetbrains.com/teamcity/2025/10/the-state-of-cicd/
  4. U. Karlovs-Karlovskis, O. Nikiforova, O. Pastor, and N. Tran, “An Industrial Evaluation of Automated CI/CD Pipeline Generation Framework,” Proceedings of the 21st International Conference on Evaluation of Novel Approaches to Software Engineering, pp. 930–937, 2026. https://dx.doi.org/10.5220/0015048200004015. [Online]. Available: https://www.scitepress.org/DigitalLibrary/Link.aspx?doi=10. 5220/0015048200004015
  5. D. C. Schmidt, “Guest Editor’s Introduction: Model-Driven Engineering,” Computer, vol. 39, no. 2, pp. 25–31, 2 2006. https://dx.doi.org/10.1109/MC.2006.58
  6. H. da Giao, V. Amaral, G. Engels, A. Flores, R. Pereira, S. Sauer, and J. Cunha, “ A Metamodel for Reengineering CI/CD Pipelines ,” in 2025 ACM/IEEE 28th International Conference on Model Driven Engineering Languages and Systems (MODELS). Los Alamitos, CA, USA: IEEE Computer Society, 10 2025. https://dx.doi.org/10.1109/MODELS67397.2025.00026 pp. 221–231. [Online]. Available: https://doi.ieeecomputersociety.org/ 10.1109/MODELS67397.2025.00026
  7. A. Flores, V. Amaral, H. Gião, and J. Cunha, “A Two-Level Model-Driven Approach for Reengineering CI/CD Pipelines.” Porto: University of Porto, 9 2024. https://dx.doi.org/10.54499/UIDB/50014/2020. [Online]. Available: https://sciproj.ptcris.pt/157676UID
  8. A. Kleppe, J. Warmer, and W. Bast, “The Model Driven Architecture: Practice and Promise. 2003,” Addison Wesley, p. 170, 2003. [Online]. Available: http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle: The+Model+Driven+Architecture:+Practice+and+Promise.+2003#6
  9. J. Whittle, J. Hutchinson, and M. Rouncefield, “The State of Practice in Model-Driven Engineering,” IEEE Software, vol. 31, no. 3, pp. 79–85, 2014. https://dx.doi.org/10.1109/MS.2013.65
  10. X. Hou, Y. Zhao, Y. Liu, Z. Yang, K. Wang, L. Li, X. Luo, D. Lo, J. Grundy, and H. Wang, “Large Language Models for Software Engineering: A Systematic Literature Review,” ACM Trans. Softw. Eng. Methodol., vol. 33, no. 8, 12 2024. https://dx.doi.org/10.1145/3695988. [Online]. Available: https://doi.org/10.1145/3695988
  11. T. Guo, X. Chen, Y. Wang, R. Chang, S. Pei, N. V. Chawla, O. Wiest, and X. Zhang, “Large Language Model Based Multi-agents: A Survey of Progress and Challenges,” in Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence. California: International Joint Conferences on Artificial Intelligence Organization, 8 2024. https://dx.doi.org/10.24963/ijcai.2024/890. ISBN 978-1-956792-04-1 pp. 8048– 8057.
  12. U. Karlovs-Karlovskis, O. Nikiforova, O. Pastor, and K. Vēveris, “MDDOAI: A Model-Driven DevOps Approach to CI/CD Automation.” Riga: IEEE, 11 2025. https://dx.doi.org/10.1109/itms67030.2025.11236630. ISSN 27716937 pp. 1–9. [Online]. Available: https://ieeexplore.ieee.org/ document/11236630
  13. J. Cabot, “Vibe Modeling: Challenges and Opportunities,” in Advances in Conceptual Modeling: ER 2025 Workshops, FCM, CMLS, LLM4Modeling, OntoCom, and QUAMES, Poitiers, France, October 20–23, 2025, Proceedings. Berlin, Heidelberg: Springer-Verlag, 2025. https://dx.doi.org/10.1007/978-3-032-08620-4_7. ISBN 978-3-032-08619-8 pp. 105– 118. [Online]. Available: https://doi.org/10.1007/978-3-032-08620-4_7
  14. A. R. Hevner, S. T. March, J. Park, and S. Ram, “Design Science in Information Systems Research,” MIS Q., vol. 28, pp. 75–105, 2004. [Online]. Available: https://api.semanticscholar.org/CorpusID:13553735
  15. U. Karlovs-Karlovskis, “Development of a Model-Driven DevOps Solution Based on Context-Engineered LLM Code Generation: PROFES Doctoral Symposium.” Salerno: Springer, Cham, 12 2025. https://dx.doi.org/10.1007/978-3-032-12092-2_10. ISBN 978-3-032-12092-2. ISSN 1611-3349 pp. 153–159. [Online]. Available: https://link.springer.com/ chapter/10.1007/978-3-032-12092-2_10
  16. J. Cabot, “Exploring the use of large language models in domain-specific language development: an experience report,” in STAF Workshops, 2025. [Online]. Available: https://api.semanticscholar.org/CorpusID: 283888266
  17. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba, “Evaluating Large Language Models Trained on Code,” 7 2021. https://dx.doi.org/10.48550/arXiv.2107.03374. [Online]. Available: http://arxiv.org/abs/2107.03374
  18. J. Rocco, D. Di Ruscio, C. Di Sipio, P. Nguyen, and R. Rubei, “On the use of large language models in model-driven engineering,” Software and Systems Modeling, vol. 24, pp. 923–948, 4 2025. https://dx.doi.org/10.1007/s10270025-01263-8
  19. S. Schulhoff, M. Ilie, N. Balepur, K. Kahadze, A. Liu, C. Si, Y. Li, A. Gupta, H. Han, S. Schulhoff, P. S. Dulepet, S. Vidyadhara, D. Ki, S. Agrawal, C. Pham, G. C. Kroiz, F. Li, H. Tao, A. Srivastava, H. D. Costa, S. Gupta, M. L. Rogers, I. Goncearenco, G. Sarli, I. Galynker, D. Peskoff, M. Carpuat, J. White, S. Anadkat, A. Hoyle, and P. Resnik, “The Prompt Report: A Systematic Survey of Prompt Engineering Techniques,” 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:276617957
  20. D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, D. Metropolitansky, R. O. Ness, and J. Larson, “From Local to Global: A Graph RAG Approach to Query-Focused Summarization,” 2 2025. [Online]. Available: http://arxiv.org/abs/2404.16130
  21. S. Hong, M. Zhuge, J. Chen, X. Zheng, Y. Cheng, C. Zhang, J. Wang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou, C. Ran, L. Xiao, C. Wu, and J. Schmidhuber, “MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework,” 11 2024. [Online]. Available: http://arxiv.org/abs/2308.00352
  22. L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica, “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena,” in Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. [Online]. Available: https://openreview.net/forum?id=uccHPGDlao