RetentionOS: A Closed-Loop Prescriptive Machine Learning Framework for Real-Time Customer Retention Smeet Patel
DOI:
https://doi.org/10.63282/3050-9262.IJAIDSML-V7I2P118Keywords:
Closed-Loop Machine Learning, Prescriptive Analytics, Uplift Modeling, Real-Time Decisioning, Customer Retention, Causal Attribution, Champion-Challenger Model Selection, Feature Store Architecture, Heterogeneous Treatment Effects, Machine Learning Operations MlopsAbstract
Existing customer retention systems in subscription-based industries rely on open-loop churn prediction that never systematically measures whether interventions caused customers to stay. This paper proposes RetentionOS, a closed-loop prescriptive machine learning framework that integrates churn propensity, pricing sensitivity, and customer lifetime value (CLV) profitability models with a low-latency feature store (AWS DynamoDB, Redis) and a tlive scoring pipeline, delivering top-3 curated retention offers via API within strict sub-second latency requirements. A champion-challenger traffic split with formal statistical promotion criteria drives continuous evidence-based model evolution. Outcome measurement via daily order processing tables, combined with holdout-based causal attribution, closes the learning loop. The framework contributes mathematical formulation of the decision objective and CLV-weighted retention metric, formal algorithms for arbitration and champion-challenger promotion, a capability comparison against four baseline approaches, and a reproducible ablation methodology. RetentionOS is proposed as a reference framework directly applicable to any high-CLV customer lifecycle requiring real-time intervention and causal learning - telecommunications retention, financial services attrition, subscription software cancellation, and healthcare patient engagement are immediate target domains.
References
[1] Huang, M. T. Kechadi, and B. Buckley, “Customer churn prediction in telecommunications,” Expert Syst. Appl., vol. 39, no. 1, pp. 1414-1425, Jan. 2012.
[2] A. De Caigny, K. Coussement, and K. W. De Bock, “A new hybrid classification algorithm for customer churn prediction based on logistic regression and decision trees,” Eur. J. Oper. Res., vol. 269, no. 2, pp. 760-772, Sep. 2018.
[3] H. Zhao, X. Yao, and R. P. W. Duin, “Predicting customer churn in telecommunications: A review,” Int. J. Inf. Manage., vol. 55, Dec. 2020, Art. no. 102185.
[4] N. J. Radcliffe and P. D. Surry, “Real-world uplift modelling with significance-based uplift trees,” Stochastic Solutions, Edinburgh, UK, White Paper TR-2011-1, 2011.
[5] S. R. Künzel, J. S. Sekhon, P. J. Bickel, and B. Yu, “Metalearners for estimating heterogeneous treatment effects using machine learning,” Proc. Nat. Acad. Sci., vol. 116, no. 10, pp. 4156-4165, Mar. 2019.
[6] P. Gutierrez and J.-Y. Gerardy, “Causal inference and uplift modelling: A review of the literature,” in Proc. 3rd Int. Conf. Predictive Appl. APIs, vol. 67, pp. 1-13, 2017.
[7] P. R. Rosenbaum and D. B. Rubin, “The central role of the propensity score in observational studies for causal effects,” Biometrika, vol. 70, no. 1, pp. 41-55, Apr. 1983.
[8] S. Athey and G. Imbens, “Recursive partitioning for heterogeneous causal effects,” Proc. Nat. Acad. Sci., vol. 113, no. 27, pp. 7353-7360, Jul. 2016.
[9] J. Hermann and M. Del Balso, “Meet Michelangelo: Uber’s machine learning platform,” Uber Engineering Blog, Sep. 2017. Accessed: Apr. 17, 2026. [Online]. Available: https://eng.uber.com/michelangelo-machine-learning-platform/
[10] M. Olson, M. Armstrong, A. Gundlach, and J. Stoica, “The feature store: Architectural patterns for machine learning,” in Proc. IEEE Int. Conf. Big Data, pp. 3420-3427, Dec. 2020.
[11] N. Polyzotis, S. Roy, S. E. Whang, and M. Zinkevich, “Data management challenges in production machine learning,” in Proc. ACM SIGMOD Int. Conf. Manage. Data, pp. 1723-1726, May 2017.
[12] R. Kohavi, A. Deng, B. Frasca, T. Walker, Y. Xu, and N. Pohlmann, “Online controlled experiments at large scale,” in Proc. 19th ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, pp. 1168-1176, Aug. 2013.
[13] O. Chapelle and L. Li, “An empirical evaluation of Thompson sampling,” in Advances in Neural Information Processing Systems 24, pp. 2249-2257, Dec. 2011.
[14] J. Gama, I. Žliobaitė, A. Bifet, M. Pechenizkiy, and A. Bouchachia, “A survey on concept drift adaptation,” ACM Comput. Surv., vol. 46, no. 4, pp. 1-37, Apr. 2014.
[15] M. Treveil, N. Omont, C. Stenac, K. Lefevre, D. Phan, J. Zentici, A. Lavoillotte, M. Miyazaki, and L. Heidmann, Introducing MLOps: How to Scale Machine Learning in the Enterprise. Sebastopol, CA, USA: O’Reilly Media, 2020.
