Enhancing data imputation in complex datasets using Lagrange polynomial interpolation and hot-deck fusion

Abstract

Data imputation is vital in preserving the quality of datasets in machine learning, where missing data leads to decreased model accuracy. This research proposes a new imputation method called Lagrange Polynomial Interpolation with Hot-Deck Fusion (LPIHD) to enhance the quality and reliability of imputed datasets, mainly when the data is multifaceted and comprises multiple types. LPIHD combines Lagrange Polynomial Interpolation and Hot-Deck Fusion. Lagrange Polynomial Interpolation estimates missing values using known data points. Hot-Deck Fusion refines these estimates by borrowing similar values from a donor population. This hybrid approach applied to two distinct datasets about wine quality and heart diseases, enhances precision by achieving lower MAE and RMSE values than those previously recorded. LPIHD achieved better accuracy for the wine quality and heart disease datasets, respectively, at varying rates of missing data. MAE and RMSE were also notably reduced across both datasets, affirming the method's efficacy. These findings suggest that LPIHD can produce better and more accurate data imputations, making it a helpful technique for the field that needs a strong analytical platform.