HUMAN - AI COLLABORATION IN SECURE SYSTEM DEVELOPMENT: DECISION PROCESSES AND ORGANIZATIONAL IMPLICATIONS
DOI:
https://doi.org/10.68302/std2026.vol3.133Keywords:
artificial intelligence (AI), decision-making processes, human - AI collaboration, organizational factors, secure system developmentAbstract
Artificial intelligence is increasingly embedded in secure software development, yet its impact on security outcomes depends less on technical capability than on the socio-technical integration of its outputs. This paper examines human - AI collaboration, focusing on the critical interface between automated decision-support tools and team-level decision-making. Adopting an interdisciplinary perspective, the study analyzes how AI-generated insights inform threat modeling, risk assessment, and security architecture design. The paper proposes the Human–AI Decision Framework (HADF) as an analytical framework for organizing the interaction between AI outputs, human expertise, team deliberation, and organizational context. It identifies key organizational moderators - including trust, explainability, and coordination - that shape whether AI enhances security or introduces new vulnerabilities through misalignment risks. In doing so, the study bridges a gap between technical AI capability and the social processes through which security decisions are actually formed. By positioning AI as a supporting actor within a dynamic organizational system, the study provides a grounded foundation for future empirical research on AI-driven security governance. The framework also offers a structured basis for operationalizing human - AI collaboration in security-critical environments, enabling more consistent evaluation and refinement of decision practices.
Downloads
References
[1] Casola, V., De Benedictis, A., Mazzocca, C., & Orbinato, V. (2024). Secure software development and testing: A model-based methodology. Computers & Security, 137, 103639. https://doi.org/10.1016/j.cose.2023.103639
[2] NIST. (2020). Security and privacy controls for information systems and organizations (SP 800-53 Rev. 5). National Institute of Standards and Technology.
[3] Sarker, I.H., Furhad, M.H. and Nowrozy, R. (2021) AI-Driven Cybersecurity: An Overview, Security Intelligence Modeling and Research Directions. SN Computer Science, 2, Article No. 173. https://doi.org/10.1007/s42979-021-00557-0
[4] Taddeo, M., & Floridi, L. (2018). Regulate artificial intelligence to avert cyber arms race. Nature, 556(7701), 296–298. https://doi.org/10.1038/d41586-018-04602-6
[5] Hamza, M., Younas, M., Hummel, M., & Sajjad, M. (2024). Human–AI collaboration in software engineering: A systematic mapping study. In Proceedings of the 17th International Conference on Cooperative and Human Aspects of Software Engineering. Association for Computing Machinery. https://doi.org/10.1145/3643690.3648236
[6] Alhumam, A., & Ahmed, S. (2024). Software requirement engineering over the federated environment in distributed software development process. Journal of King Saud University – Computer and Information Sciences, 36(9), 102201. https://doi.org/10.1016/j.jksuci.2024.102201
[7] Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., Teevan, J., Kikin-Gil, R., & Horvitz, E. (2019). Guidelines for human-AI interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, Article 3, 13 pages. https://doi.org/10.1145/3290605.3300233
[8] Shneiderman, B. (2020). Human-centered artificial intelligence: Reliable, safe & trustworthy. International Journal of Human–Computer Interaction, 36(6), 495–504. https://doi.org/10.1080/10447318.2020.1741118
[9] Snyder, H. (2024). Designing the literature review for a strong contribution. Journal of Decision Systems, 33(4), 551–558. https://doi.org/10.1080/12460125.2023.2197704
[10] Papagiannidis, E., Mikalef, P., & Conboy, K. (2025). Responsible artificial intelligence governance: A review and research framework. The Journal of Strategic Information Systems, 34(2), 101885. https://doi.org/10.1016/j.jsis.2024.101885
[11] Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 30(3), 286–297. https://doi.org/10.1109/3468.844354
[12] Klein, G. (2008). Naturalistic decision making. Human Factors, 50(3), 456–460. https://doi.org/10.1518/001872008X288385
[13] Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1), 32–64. https://doi.org/10.1518/001872095779049543
[14] Lee, J. D., & See, K. A. (2004) Trust in Automation: Designing for Appropriate Reliance. Human Factors: The Journal of the Human Factors and Ergonomics Society, 46, 50-80. https://doi.org/10.1518/hfes.46.1.50_30392
[15] Glikson, E., & Woolley, A. W. (2020). Human trust in artificial intelligence: Review of empirical research. Academy of Management Annals, 14(2), 627–660. https://doi.org/10.5465/annals.2018.0057
[16] Jarrahi, M. H. (2018). Artificial intelligence and the future of work: Human-AI symbiosis in organizational decision making. Business Horizons, 61(4), 577–586. https://doi.org/10.1016/j.bushor.2018.03.007
[17] Kellogg, K. C., Valentine, M. A., & Christin, A. (2020). Algorithms at work: The new contested terrain of control. Academy of Management Annals, 14(1), 366–410. https://doi.org/10.5465/annals.2018.0174
[18] Rai, A. (2020). Explainable AI: From black box to glass box. Journal of the Academy of Marketing Science, 48(1), 137–141. https://doi.org/10.1007/s11747-019-00710-5
[19] Narleva K., Narlev Y., Gancheva Y.,, (2023). Maritime Education and New Skills in the Digitisation Process, HR and Technologies, Creative Space Association, 2, 57-72.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Marina Marinova-Stoyanova, Boris Gramchev
This work is licensed under a Creative Commons Attribution 4.0 International License.
