Cognitive Load Reduction in On-Call Rotations via Predictive Alert Severity Scoring Using Machine Learning in Financial Cloud Operations

Abstract

Alert fatigue in cloud-native site reliability engineering represents a systemic threat to both operational reliability and engineer wellbeing. In high-availability financial services environments, on-call engineers receive hundreds of automated alerts per month from integrated observability platforms, of which a substantial fraction are non-actionable noise. This paper presents an ML-driven alert severity scoring framework deployed in production across six credit union banking applications, designed to reduce the cognitive load of on-call engineers by intelligently triaging PagerDuty alert streams before human engagement. The framework trains a gradient-boosted classifier on over 50,000 historical alert events sourced from Dynatrace Davis AI and PagerDuty, engineering 17 features capturing service dependency depth, deployment recency, historical false positive rates, time-of-day patterns, and cross-source metric correlations. Evaluated through a 30-day shadow validation protocol against actual engineer triage decisions, the classifier achieved 89.3% severity classification accuracy with a 2.1% P1 miss rate. Live deployment produced a 34% reduction in actionable alert volume, a 41% reduction in after-hours pages, and a 28% improvement in P1 mean time to acknowledge. A structured engineer wellbeing measurement program documented improvement in post-shift satisfaction scores from 5.8 to 7.4 out of 10 following system activation. This paper presents the feature engineering methodology, the shadow validation protocol, the deployment architecture within a SOC 2 regulated banking environment, and the case for treating engineer wellbeing as a first-class reliability metric.