A robust feature selection approach for high-dimensional medical data classification using enhanced correlation attribute evaluation

Abstract

The challenge of high-dimensional feature spaces and redundant attributes significantly impacts classification performance in medical datasets. Addressing this, the proposed Enhanced Correlation Attribute Evaluation (E-CAE) method effectively integrates multiple correlation measures such as Pearson, Spearman, Kendall, Biweight Midcorrelation, and Distance Correlation to rank and select the most relevant features. This hybrid feature selection technique was rigorously tested on three datasets: the Darwin dataset, Parkinson’s speech dataset, and Dyslexia dataset. The E-CAE method demonstrated superior classification performance across various models, achieving a remarkable 95.64% accuracy on the Darwin dataset, 93.42% accuracy on the Parkinson’s dataset, and 90.86% accuracy on the Dyslexia dataset. These results notably outperformed traditional feature selection techniques. The novelty of this approach lies in its composite scoring mechanism, which ensures robust feature evaluation and significantly enhances classification accuracy across diverse biomedical datasets.