COMPARING FEATURE SELECTION METHODS AND EXPLAINABLE AI TECHNIQUES
Keywords:
feature selection, explainable artificial intelligence, SHAP, permutation importance, medical machine learning, interpretable classification, Jaccard similarityAbstract
Feature selection and explainable artificial intelligence (XAI) are both used to improve the interpretability of medical machine learning, but they answer different questions. Feature selection asks which variables should be retained before training, whereas XAI asks which variables influenced an already trained model. This study compares six feature selection methods (ANOVA F-test, mutual information, RFE, L1 logistic regression, random forest importance, and extra trees importance) with three XAI views (SHAP values, permutation importance, and model-based random forest importance). Experiments were performed in Python on Breast Cancer Wisconsin Diagnostic, Pima Indians Diabetes, and Heart Disease Cleveland datasets. Logistic regression, random forest, SVM, and gradient boosting were evaluated by accuracy, precision, recall, F1-score, and ROC-AUC. Agreement between selected and explained top features was measured with Jaccard similarity and overlap percentage. The best full-feature ROC-AUC values were 0.998, 0.823, and 0.862 for the three datasets, respectively. Mean Jaccard agreement was moderate and dataset-dependent: 0.612 for Breast Cancer, 0.630 for Pima Diabetes, and 0.577 for Heart Disease. The results show that feature selection and XAI are complementary rather than interchangeable.References
[1] Sadeghi, Z., Alizadehsani, R., Cifci, M. A., Kausar, S., Rehman, R., Mahanta, P., et al. (2024). A review of Explainable Artificial Intelligence in healthcare. Computers and Electrical Engineering, 118, 109370. https://doi.org/10.1016/j.compeleceng.2024.109370
[2] Zhang, K., et al. (2026). A comprehensive review of explainable artificial intelligence in healthcare: methods, evaluation, and clinical integration. iScience. https://doi.org/10.1016/j.isci.2026.115026
[3] Pudjihartono, N., Fadason, T., Kempa-Liehr, A. W., & O'Sullivan, J. M. (2022). A Review of Feature Selection Methods for Machine Learning-Based Disease Risk Prediction. Frontiers in Bioinformatics, 2, 927312. https://doi.org/10.3389/fbinf.2022.927312
[4] Dunn, J., Mingardi, L., & Zhuo, Y. D. (2021). Comparing interpretability and explainability for feature selection. arXiv:2105.05328.
[5] Zacharias, J., et al. (2022). Designing a feature selection method based on explainable artificial intelligence. Electronic Markets. https://doi.org/10.1007/s12525-022-00608-1
[6] Kraev, E., Koseoglu, B., Traverso, L., & Topiwalla, M. (2024). shap-select: Lightweight Feature Selection Using SHAP Values and Regression. arXiv:2410.06815.
[7] Salih, A., Raisi-Estabragh, Z., Galazzo, I. B., Radeva, P., Petersen, S. E., Menegaz, G., & Lekadir, K. (2023). A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME. arXiv:2305.02012.
[8] Ponce-Bobadilla, A. V., et al. (2024). Practical guide to SHAP analysis: Explaining supervised machine learning model predictions in drug development. Clinical and Translational Science.
[9] scikit-learn developers. Feature selection. scikit-learn User Guide. https://scikit-learn.org/stable/modules/feature_selection.html
[10] scikit-learn developers. Permutation feature importance. scikit-learn User Guide. https://scikit-learn.org/stable/modules/permutation_importance.html
[11] UCI Machine Learning Repository. Breast Cancer Wisconsin (Diagnostic) dataset. https://archive.ics.uci.edu/dataset/17/breast+cancer+wisconsin+diagnostic
[12] UCI Machine Learning Repository. Heart Disease dataset. https://archive.ics.uci.edu/dataset/45/heart+disease
[13] Kaggle / UCI ML. Pima Indians Diabetes Database. https://www.kaggle.com/datasets/uciml/pima-indians-diabetes-database
[14] Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30.