AI-Powered Anomaly Detection for Real-Time Financial Platform Reliability Monitoring: Benchmarking Machine Learning Algorithms for Detecting Outages, Latency Spikes, and Security Violations in Banking
DOI:
https://doi.org/10.63282/3050-9262.IJAIDSML-V4I3P116Keywords:
Aiops, Anomaly Detection, Devops, SRE, Observability, Banking Systems, Reliability Monitoring, Latency, Outage Detection, Security Analytics, Machine Learning BenchmarkingAbstract
Financial banking platforms operate under stringent reliability and security requirements while continuously evolving through frequent releases, configuration changes, and infrastructure modernization. Traditional monitoring approaches—static thresholds, handcrafted alert rules, and siloed dashboards—struggle to keep pace with the scale, seasonality, and dependency complexity of modern banking systems. This study provides a practitioner-oriented manuscript on AI-powered anomaly detection for real-time reliability monitoring in financial platforms. Rather than treating anomaly detection as a standalone ML exercise, we treat it as a production capability embedded in DevOps and Site Reliability Engineering (SRE). We outline a reference AIOps architecture shaped by banking constraints—auditability, model governance, access control, and controlled data handling. We then benchmark representative detectors (robust statistics, classical machine learning, and deep learning) against three high-impact categories of production risk: outages, latency spikes (including tail latency degradation), and security violations. Evaluation prioritizes operational measures—time-to-detect, false alerts per service-day, robustness under seasonality, explainability, and cost-to-operate—alongside conventional precision/recall.
References
[1] M. Goldstein and S. Uchida, "A comparative evaluation of unsupervised anomaly detection algorithms for multivariate data," PLOS ONE, vol. 11, no. 4, pp. 1–31, 2016.
[2] F. T. Liu, K. M. Ting, and Z.-H. Zhou, "Isolation forest," in Proc. IEEE Int. Conf. Data Mining (ICDM), Miami, FL, USA, 2008, pp. 413–422.
[3] B. Schölkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson, "Estimating the support of a high-dimensional distribution," Neural Computation, vol. 13, no. 7, pp. 1443–1471, 2001.
[4] S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.
[5] R. P. Adams and D. J. C. MacKay, "Bayesian online changepoint detection," arXiv:0710.3742, 2007.
[6] D. P. Kingma and M. Welling, "Auto-encoding variational Bayes," in Proc. Int. Conf. Learning Representations (ICLR), San Diego, CA, USA, 2014.
[7] A. Agade and S. Balpande, “Exploring the Non-Medical impacts of Covid-19 using Natural Language Processing,” International Journal of Computer Applications, vol. 175, no. 36, pp. 16–23, Dec. 2020, doi: 10.5120/ijca2020920923.
[8] A. Paszke et al., "PyTorch: An imperative style, high-performance deep learning library," in Advances in Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada, 2019.
[9] C. M. Bishop, Pattern Recognition and Machine Learning. New York, NY, USA: Springer, 2006.
[10] N. G. Leveson, Engineering a Safer World: Systems Thinking Applied to Safety. Cambridge, MA, USA: MIT Press, 2011.
[11] B. Beyer et al., Site Reliability Engineering: How Google Runs Production Systems. Sebastopol, CA, USA: O’Reilly Media, 2016.
[12] M. Chen, A. S. Bhandari, and J. Li, "The emergence of AIOps: Opportunities and challenges," IEEE Access, vol. 8, pp. 179698–179712, 2020.
[13] J. Postel and J. Reynolds, "Common event expression (CEE)," Internet Engineering Task Force (IETF), RFC 8425, 2018.
