Multi-Metric Evaluation Framework for Machine Learning-Based Load Prediction in e-Governance Systems

Abstract

The explosive growth of e-Governance platforms will necessitate transitioning from lifecycle reactive handling to proactive rather than just reactionary methods for handling e-Governance workloads and therefore managing resources effectively. Given that e-Governance workloads consist of highly dynamic content, load predictions must be sufficiently accurate for efficient resource selection and provisioning, continual discussion between workloads that need to comply with SLAs, and enabling the systematic handling of e-Governance workload. Machine learning-based approaches will provide strong predictive capabilities; however, careful consideration must be given to how those ML-based approaches will be deployed into the environment of an e-Governance system with regards to predictive accuracy, computational performance, scalability and robustness. This research paper will present a complete multi-metric evaluation framework that was developed to assess Load Prediction Models for e-Governance Platforms. The evaluation framework will consist of regressors, including Linear Regression, Instance-Based Learning, and Ensemble Approaches such as Random Forest, Gradient Boosting, XGBoost, LightGBM and CatBoost; however, when conducting the evaluation of each of the regression models it should not only include the traditional manner of evaluating for accuracy but also include training time, prediction latency, amount of Memory consumed for model training, amount of Data Inference Processed, Worst Case Error Percentiles, and Scalable Assessment of All Proposed Regression Models with respect to Data Size. The experimental results show that both Ensemble and Gradient Boosting Models significantly outperform conventional Baseline Approaches in terms of the Accuracy of the Prediction of the Response Variable. By combining the various advantages of all models tested and evaluating them based upon completed multi-metric evaluation framework, LightGBM has the overall best combination of Accuracy, Scalable Assessment, High Inference Efficiency and Lower Memory Usage. The results of this study provide insights into practical aspects of deploying intelligent load prediction solutions designed to improve the performance and reliability of e-Governance platforms.