Autonomous DevOps Pipelines Using Multi-Agent Systems for Enterprise-Grade Cloud Delivery
DOI:
https://doi.org/10.63282/3050-9262.IJAIDSML-V5I1P117Keywords:
Autonomous DevOps Pipelines, Multi-Agent Systems (MAS), AI-Driven DevOps (AIOps), Cloud-Native Continuous Delivery, Enterprise-Grade Cloud Delivery, Self-Orchestrating CI/CD, Distributed Intelligent Systems, Self-Healing Cloud SystemsAbstract
Traditional DevOps pipeline models are being stretched to their limits due to the increasing complexity and variability of cloud-native and multi-cloud environments. While many CI/CD pipelines have an automated process, they typically use a central orchestrator, rule-based systems, and manual intervention by humans. This makes them less capable of adapting to runtime uncertainty, cascade failures, and emerging security threats. This research will develop the concept of Autonomous DevOps Pipelines based on Multi-Agent Systems (MAS) as a foundation for large-scale cloud delivery across enterprises. The primary objective of this research is to explore the extent to which intelligent software agents can be used throughout the DevOps lifecycle to provide self-orchestration, adaptive decision making, and continuous optimization within changing operational environments. Using a design-oriented and conceptual research methodology, this paper brings together prior research on DevOps automation, Distributed Artificial Intelligence, and AIOps; it then unifies the findings in a reference architecture for a multi-agent system in DevOps. The proposed reference architecture defines the roles of agents, the processes through which agents will coordinate with each other, how agents will learn from interactions with their environment, and how agents will be governed in terms of policies that reflect the needs of an enterprise. The evaluation of the reference architecture demonstrates that MAS-based DevOps pipelines can improve deployment reliability, decrease mean time to recovery, and accelerate release cycles while maintaining security, regulatory compliance, and auditability. The paper also identifies significant research and engineering challenges associated with MAS-based DevOps pipelines including, but not limited to, the cost of coordinating between agents, the trustworthiness and explainability of autonomous agent decisions, and the management of operational risk. Ultimately, the evaluation of the reference architecture suggests that using multiple intelligent agents provides a viable and scalable means to enable self-healing, self-optimizing, and intelligent cloud delivery platforms that can support complex enterprise environments.
References
[1] C. Collins, D. Dennehy, K. Conboy, and P. Mikalef, “Artificial Intelligence in Information Systems research: a Systematic Literature Review and Research Agenda,” International Journal of Information Management, vol. 60, no. 102383, 2021, Available: https://www.sciencedirect.com/science/article/pii/S0268401221000761
[2] Josu Díaz-de-Arcaya, A. I. Torre-Bastida, Gorka Zárate, Raúl Miñón, and A. Almeida, “A Joint Study of the Challenges, Opportunities, and Roadmap of MLOps and AIOps: A Systematic Survey,” ACM Computing Surveys, vol. 56, no. 4, pp. 1–30, Oct. 2023, doi: https://doi.org/10.1145/3625289.
[3] M. Steidl, M. Felderer, and R. Ramler, “The pipeline for the continuous development of artificial intelligence models—Current state of research and practice,” Journal of Systems and Software, p. 111615, Jan. 2023, doi: https://doi.org/10.1016/j.jss.2023.111615.
[4] I. Karamitsos, S. Albarhami, and C. Apostolopoulos, “Applying DevOps Practices of Continuous Automation for Machine Learning,” Information, vol. 11, no. 7, p. 363, Jul. 2020, doi: https://doi.org/10.3390/info11070363.
[5] C. Pahl, P. Jamshidi, and O. Zimmermann, “Architectural Principles for Cloud Software,” ACM Transactions on Internet Technology, vol. 18, no. 2, pp. 1–23, Mar. 2018, doi: https://doi.org/10.1145/3104028.
[6] Q. Cheng et al., “AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges,” arXiv.org, Apr. 10, 2023. https://arxiv.org/abs/2304.04661
[7] “Applying DevOps Practices of Continuous Automation for Machine Learning - ProQuest,” Proquest.com, 2020. https://www.proquest.com/openview/d9ae0de547805e8f6d1be658b52c3647/1?pq-origsite=gscholar&cbl=2032384
[8] M. Shahin, M. Ali Babar, and L. Zhu, “Continuous Integration, Delivery and Deployment: a Systematic Review on Approaches, Tools, Challenges and Practices,” IEEE Access, vol. 5, pp. 3909–3943, 2017, doi: https://doi.org/10.1109/access.2017.2685629.
[9] F. M. A. Erich, C. Amrit, and M. Daneva, “A qualitative study of DevOps usage in practice,” Journal of Software: Evolution and Process, vol. 29, no. 6, p. e1885, Jun. 2017, doi: https://doi.org/10.1002/smr.1885.
[10] B. Porter, R. R. Filho, and P. Dean, “A Survey of Methodology in Self-Adaptive Systems Research,” Lancaster EPrints (Lancaster University), pp. 168–177, Aug. 2020, doi: https://doi.org/10.1109/acsos49614.2020.00039.
