TALEX: Transformer-Attention-Led EXplainable Feature Selection for Sentiment Classification

Abstract

Feature selection plays a crucial role in sentiment analysis, especially in transformer-based architecture where large and complex feature spaces often hinder both efficiency and interpretability. Conventional statistical and heuristic selection methods fail to fully exploit transformer attention signals and typically lack faithfulness to the model’s actual decision process. This research introduces TALEX, a Transformer-Attention-Led EXplainable Feature Selection framework, designed to derive compact, discriminative, and interpretable feature subsets for sentiment classification. TALEX integrates multi-view saliency signals from transformer attention, Integrated Gradients, and SHAP to rank features, followed by differentiable gating optimized with explainability-alignment loss. Extensive experiments on four benchmark datasets: MR, CR, IMDB, and SemEval 2013, demonstrate that TALEX achieves competitive or superior accuracy while reducing feature dimensionality by 30–60%. Furthermore, deletion–insertion analyses and attribution alignment confirm high faithfulness and explanation stability. By aligning feature selection with explanation mechanisms, TALEX effectively bridges the gap between model efficiency and interpretability, providing a transparent and scalable foundation for real-world sentiment analysis applications.