Beyond Final Scores: Identifying Behavioural Decision-Making Profiles from Cybersecurity Simulation Log Data

Allison, Jordan ORCID: https://orcid.org/0000-0001-8513-4646 and Adihetty, Hasini (2026) Beyond Final Scores: Identifying Behavioural Decision-Making Profiles from Cybersecurity Simulation Log Data. In: 6th International Conference on Educational Technology and Online Learning (ICETOL) Proceedings. ICETOL. (In Press)

Text 16627 Allison, J and Adihetty, H. (2026) Beyond Final Scores Identifying Behavioural Decision-Making Profiles from Cybersecurity Simulation Log Data.pdf - Accepted Version Restricted to Repository staff only Available under License Creative Commons Attribution 4.0. Download (508kB)

Abstract

Simulation-based learning provides opportunities for learners to practise complex decision-making in authentic contexts while generating rich behavioural data that extend beyond traditional performance measures. In cybersecurity education, although technical laboratories and capture-the-flag activities develop important practical skills, they do not always represent the organisational, ethical, financial and human pressures involved in cybersecurity management. Learning analytics has traditionally focused on performance outcomes, yet these measures often reveal little about the decision-making processes that underpin learner behaviour. To address this challenge, a browser-based cybersecurity management simulation was developed for Level 6 and Level 7 cybersecurity students. The simulation places learners in the role of a cybersecurity manager and presents 15 organisational incidents, each offering three possible responses with consequences for security, budget, reputation, staff morale and board happiness. Each playthrough generates a decision log that records the sequence of choices made, providing a detailed account of how learners respond to risk and balance competing priorities over time. This study examines how these decision logs can be used within learning analytics to identify decision-making strategies that cannot be understood from final scores alone, thereby providing a richer understanding of learners' decision-making processes within simulation-based cybersecurity education. Following data preparation and validation, 136 playthroughs were available for analysis, including 2,009 decision-level observations. Of these, 125 included all 15 scenarios, while 11 ended early because the simulated organisation became bankrupt. These early-ending runs were retained as meaningful gameplay outcomes. To move from this cleaned dataset to an analysis of learner strategies, a theory-informed coding framework classified the 45 available decision options across four behavioural dimensions: decision orientation, risk management approach, human-element governance, and transparency strategy. These codes were then linked to the decision logs, allowing each playthrough to be represented as a proportional behavioural profile. Because the number of recorded decisions differed between complete and early ending runs, proportions were used instead of raw counts. Final simulation scores were deliberately excluded from the clustering process to ensure that the resulting groups reflected patterns of decision-making rather than performance outcomes. Multiple clustering approaches, including k-means, agglomerative clustering, Gaussian mixture models, spectral clustering and DBSCAN were compared using silhouette scores, cluster-size distributions and interpretability. K-means with k = 4 was selected as the optimal solution. This solution produced four behavioural decision-making profiles: Adaptive Balancers, Detached Operators, Surveillance Regulators and Protective Controllers. These profiles differed in risk tolerance, security prioritisation, operational focus, staff governance, monitoring and transparency, and were associated with different final outcome patterns. By transforming raw decision logs into interpretable behavioural profiles, the analysis revealed broader decision-making strategies that final scores alone could not capture. In this way, the study contributes to learning analytics by transforming decision logs into interpretable evidence that can support structured reflection, personalised feedback, adaptive simulation design, and the continued development of simulation-based cybersecurity learning. More broadly, the findings demonstrate the value of behavioural profiling for understanding complex decision-making in educational simulations, and provide a foundation for future research into learner modelling within AI-supported and simulation-based learning environments.

Item Type:	Book Section
Uncontrolled Keywords:	Simulation-based learning; Learning analytics; Cluster analysis; Decision-making; Cybersecurity education
Related URLs:	Organisation
Subjects:	Q Science > QA Mathematics > QA76 Computer software > QA76.9 Other topics > QA76.9.B45 Big data Q Science > QA Mathematics > QA76 Computer software > QA76.9 Other topics > QA76.9.H85 Human-computer interaction Q Science > QA Mathematics > QA76 Computer software > QA76.9 Other topics > QA76.9.V5 Virtual computer systems Q Science > QA Mathematics > QA76 Computer software > QA76.9 Other topics > QA76.9.C65 Computer simulation
Divisions:	Schools and Research Institutes > School of Business, Computing and Social Sciences
Depositing User:	Kamila Niekoraniec
Date Deposited:	05 Nov 2026 11:42
Last Modified:	08 Nov 2026 09:30
URI:	https://eprints.glos.ac.uk/id/eprint/16627

University Staff: Request a correction | Repository Editors: Update this record