Dual-Phase Local Search Embedded Multi-Verse Optimizer for Optimal Feature Subset Selection

Document Type : Research Paper

Authors

1 Ph.D. Candidate, Department of Industrial Engineering, K. N. Toosi University of Technology, Tehran, Iran.

2 Associate Professor, Department of Industrial Engineering, K.N. Toosi University of Technology, Tehran, Iran.

Abstract

Feature selection plays a pivotal role in enhancing the performance of machine learning models by reducing dimensionality, improving interpretability, and minimizing computational overhead. This study presents the Improved Multi-Verse Optimizer (IMVO), a new feature selection algorithm that merges the global exploration ability of the standard Multi-Verse Optimizer (MVO) with a dual-phase mutation-based local search mechanism. Unlike previous MVO-based or hybrid metaheuristic approaches, IMVO simultaneously strengthens exploitation through targeted refinement of the best solutions and preserves population diversity via periodic random-solution mutations. This strategic combination mitigates premature convergence, accelerates convergence speed, and improves robustness across diverse high-dimensional datasets. Comprehensive experiments on 14 widely used datasets obtained from the UCI repository show that IMVO consistently achieves higher classification accuracy, fewer selected features, and lower fitness values than five state-of-the-art algorithms (MVO, GA, PSO, SSA, HHO). Quantitative analysis using the Wilcoxon signed-rank test certifies the significance of these enhancements, underscoring the algorithm’s reliability. While the inclusion of local search increases computational cost, the demonstrated gains in accuracy, stability, and feature reduction affirm this cost-benefit relationship, positioning IMVO as a competitive and versatile tool for feature selection and related optimization problems.

Keywords

Main Subjects

References

[1] J. R. Vergara and P. A. Estévez, “A review of feature selection methods based on mutual information,” Neural Comput. Appl., vol. 24, no. 1, pp. 175–186, Jan. 2014, doi: 10.1007/S00521-013-1368-0/METRICS.
[2] M. DASH and H. LIU, “Feature selection for classification,” Intell. Data Anal., vol. 1, no. 1–4, pp. 131–156, Jan. 1997, doi: 10.1016/S1088-467X (97)00008-5.
[3] R. Sihwail, K. Omar, K. A. Z. Ariffin, and M. Tubishat, “Improved Harris Hawks Optimization Using Elite Opposition-Based Learning and Novel Search Mechanism for Feature Selection,” IEEE Access, vol. 8, pp. 121127–121145, 2020, doi: 10.1109/ACCESS.2020.3006473.
[4] K. H. Sheikh et al., “EHHM: Electrical harmony based hybrid meta-heuristic for feature selection,” IEEE Access, vol. 8, pp. 158125–158141, 2020, doi: 10.1109/ACCESS.2020.3019809.
[5] B. Xue, M. Zhang, W. N. Browne, and X. Yao, “A Survey on Evolutionary Computation Approaches to Feature Selection,” IEEE Trans. Evol. Comput., vol. 20, no. 4, pp. 606–626, 2016, doi: 10.1109/TEVC.2015.2504420.
[6] M. Du, K. Wang, Z. Xia, and Y. Zhang, “Differential Privacy Preserving of Training Model in Wireless Big Data with Edge Computing,” IEEE Trans. Big Data, vol. 6, no. 2, pp. 283–295, 2018, doi: 10.1109/tbdata.2018.2829886.
[7] F. Peres and M. Castelli, “Combinatorial Optimization Problems and Metaheuristics: Review, Challenges, Design, and Development,” Appl. Sci., vol. 11, no. 14, p. 6449, Jul. 2021, doi: 10.3390/app11146449.
[8] D. H. Wolpert and W. G. Macready, “No free lunch theorems for optimization,” IEEE Trans. Evol. Comput., vol. 1, no. 1, pp. 67–82, 1997, doi: 10.1109/4235.585893.
[9] Z. Sadeghian, E. Akbari, H. Nematzadeh, and H. Motameni, “A review of feature selection methods based on meta-heuristic algorithms,” J. Exp. Theor. Artif. Intell., 2023, doi: 10.1080/0952813X.2023.2183267.
[10]         H. Rao et al., “Feature selection based on artificial bee colony and gradient boosting decision tree,” Appl. Soft Comput. J., vol. 74, pp. 634–642, 2019, doi: 10.1016/j.asoc.2018.10.036.
[11]         Z. M. Elgamal, N. B. M. Yasin, M. Tubishat, M. Alswaitti, and S. Mirjalili, “An improved harris hawks optimization algorithm with simulated annealing for feature selection in the medical field,” IEEE Access, vol. 8, pp. 186638–186652, 2020, doi: 10.1109/ACCESS.2020.3029728.
[12]         M. Tubishat, N. Idris, L. Shuib, M. A. M. Abushariah, and S. Mirjalili, “Improved Salp Swarm Algorithm based on opposition based learning and novel local search algorithm for feature selection,” Expert Syst. Appl., vol. 145, 2020, doi: 10.1016/j.eswa.2019.113122.
[13]         N. Neggaz, E. H. Houssein, and K. Hussain, “An efficient henry gas solubility optimization for feature selection,” Expert Syst. Appl., vol. 152, 2020, doi: 10.1016/j.eswa.2020.113364.
[14]         P. Hu, J. S. Pan, and S. C. Chu, “Improved Binary Grey Wolf Optimizer and Its application for feature selection,” Knowledge-Based Syst., vol. 195, 2020, doi: 10.1016/j.knosys.2020.105746.
[15]         F. Kılıç, Y. Kaya, and S. Yildirim, “A novel multi population based particle swarm optimization for feature selection,” Knowledge-Based Syst., vol. 219, 2021, doi: 10.1016/j.knosys.2021.106894.
[16]         H. Alazzam, A. Sharieh, and K. E. Sabri, “A feature selection algorithm for intrusion detection system based on Pigeon Inspired Optimizer,” Expert Syst. Appl., vol. 148, 2020, doi: 10.1016/j.eswa.2020.113249.
[17]         M. Abdel-Basset, G. Manogaran, D. El-Shahat, and S. Mirjalili, “A hybrid whale optimization algorithm based on local search strategy for the permutation flow shop scheduling problem,” Futur. Gener. Comput. Syst., vol. 85, pp. 129–145, 2018, doi: 10.1016/j.future.2018.03.020.
[18]         M. Tubishat, M. Alswaitti, S. Mirjalili, M. A. Al-Garadi, M. T. Alrashdan, and T. A. Rana, “Dynamic butterfly optimization algorithm for feature selection,” IEEE Access, vol. 8, pp. 194303–194314, 2020, doi: 10.1109/ACCESS.2020.3033757.
[19]         M. D. Toksari, “A hybrid algorithm of Ant Colony Optimization (ACO) and Iterated Local Search (ILS) for estimating electricity domestic consumption: Case of Turkey,” Int. J. Electr. Power Energy Syst., vol. 78, pp. 776–782, 2016, doi: 10.1016/j.ijepes.2015.12.032.
[20]         C. Yan, J. Ma, H. Luo, and A. Patel, “Hybrid binary Coral Reefs Optimization algorithm with Simulated Annealing for Feature Selection in high-dimensional biomedical datasets,” Chemom. Intell. Lab. Syst., vol. 184, pp. 102–111, 2019, doi: 10.1016/j.chemolab.2018.11.010.
[21]         M. Shehab, A. T. Khader, M. A. Al-Betar, and L. M. Abualigah, “Hybridizing cuckoo search algorithm with hill climbing for numerical optimization problems,” ICIT 2017 - 8th Int. Conf. Inf. Technol. Proc., pp. 36–43, 2017, doi: 10.1109/ICITECH.2017.8079912.
[22]         S. Mirjalili, S. M. Mirjalili, and A. Hatamlou, “Multi-Verse Optimizer: a nature-inspired algorithm for global optimization,” Neural Comput. Appl., vol. 27, no. 2, pp. 495–513, 2016, doi: 10.1007/s00521-015-1870-7.
[23]         M. Tubishat, Z. Rawshdeh, H. Jarrah, Z. M. Elgamal, A. Elnagar, and M. T. Alrashdan, “Dynamic generalized normal distribution optimization for feature selection,” Neural Comput. Appl., vol. 34, no. 20, pp. 17355–17370, 2022, doi: 10.1007/s00521-022-07398-9.
[24]         M. M. Mafarja and S. Mirjalili, “Hybrid Whale Optimization Algorithm with simulated annealing for feature selection,” Neurocomputing, vol. 260, pp. 302–312, 2017, doi: 10.1016/j.neucom.2017.04.053.
[25]         Q. Al-Tashi, S. J. Abdul Kadir, H. M. Rais, S. Mirjalili, and H. Alhussian, “Binary Optimization Using Hybrid Grey Wolf Optimization for Feature Selection,” IEEE Access, vol. 7, pp. 39496–39508, 2019, doi: 10.1109/ACCESS.2019.2906757.
[26]         E. Emary, H. M. Zawbaa, and A. E. Hassanien, “Binary ant lion approaches for feature selection,” Neurocomputing, vol. 213, pp. 54–65, 2016, doi: 10.1016/j.neucom.2016.03.101.
[27]         S. B. Imandoust and M. Bolandraftar, “Application of K-Nearest Neighbor (KNN) Approach for Predicting Economic Events: Theoretical Background,” Int. J. Eng. Res. Appl., vol. 3, no. 5, pp. 605–610, 2013, Accessed: Jul. 19, 2024. [Online]. Available: www.ijera.com
[28]         L. Wang, L. Khan, and B. Thuraisingham, “An effective evidence theory based K-nearest neighbor (KNN) classification,” Proc. - 2008 IEEE/WIC/ACM Int. Conf. Web Intell. WI 2008, pp. 797–801, 2008, doi: 10.1109/WIIAT.2008.411.
[29]         K. Hussain, N. Neggaz, W. Zhu, and E. H. Houssein, “An efficient hybrid sine-cosine Harris hawks optimization for low and high-dimensional feature selection,” Expert Syst. Appl., vol. 176, p. 114778, Aug. 2021, doi: 10.1016/J.ESWA.2021.114778.
[30]         J. Too and S. Mirjalili, “A Hyper Learning Binary Dragonfly Algorithm for Feature Selection: A COVID-19 Case Study,” Knowledge-Based Syst., vol. 212, p. 106553, Jan. 2021, doi: 10.1016/J.KNOSYS.2020.106553.
[31]         I. Aljarah, M. Mafarja, A. A. Heidari, H. Faris, Y. Zhang, and S. Mirjalili, “Asynchronous accelerating multi-leader salp chains for feature selection,” Appl. Soft Comput., vol. 71, pp. 964–979, Oct. 2018, doi: 10.1016/J.ASOC.2018.07.040.
[32]         M. Mafarja et al., “Evolutionary Population Dynamics and Grasshopper Optimization approaches for feature selection problems,” Knowledge-Based Syst., vol. 145, pp. 25–45, Apr. 2018, doi: 10.1016/J.KNOSYS.2017.12.037.
[33]         M. M. Mafarja and S. Mirjalili, “Hybrid Whale Optimization Algorithm with simulated annealing for feature selection,” Neurocomputing, vol. 260, pp. 302–312, Oct. 2017, doi: 10.1016/J.NEUCOM.2017.04.053.
[34]         M. Tubishat, Z. Rawshdeh, H. Jarrah, Z. M. Elgamal, A. Elnagar, and M. T. Alrashdan, “Dynamic generalized normal distribution optimization for feature selection,” Neural Comput. Appl., vol. 34, no. 20, pp. 17355–17370, Oct. 2022, doi: 10.1007/s00521-022-07398-9.
[35]         M. Tubishat et al., “Dynamic Salp swarm algorithm for feature selection,” Expert Syst. Appl., vol. 164, p. 113873, Feb. 2021, doi: 10.1016/j.eswa.2020.113873.
[36]         A. E. Hegazy, M. A. Makhlouf, and G. S. El-Tawel, “Improved salp swarm algorithm for feature selection,” J. King Saud Univ. - Comput. Inf. Sci., vol. 32, no. 3, pp. 335–344, Mar. 2020, doi: 10.1016/j.jksuci.2018.06.003.
[37]         A. E. Hegazy, M. A. Makhlouf, and G. S. El-Tawel, “Feature Selection Using Chaotic Salp Swarm Algorithm for Data Classification,” Arab. J. Sci. Eng., vol. 44, no. 4, pp. 3801–3816, Apr. 2019, doi: 10.1007/s13369-018-3680-6.
[38]         S. Arora and P. Anand, “Binary butterfly optimization approaches for feature selection,” Expert Syst. Appl., vol. 116, pp. 147–160, Feb. 2019, doi: 10.1016/j.eswa.2018.08.051.
[39]         T. Thaher, A. A. Heidari, M. Mafarja, J. S. Dong, and S. Mirjalili, “Binary Harris Hawks Optimizer for High-Dimensional, Low Sample Size Feature Selection,” 2020, pp. 251–272. doi: 10.1007/978-981-32-9990-0_12.
[40]         E. BAŞ and E. ÜLKER, “An efficient binary social spider algorithm for feature selection problem,” Expert Syst. Appl., vol. 146, p. 113185, May 2020, doi: 10.1016/j.eswa.2020.113185.
Advances in Industrial Engineering
Volume 59, Issue 2
December 2025
Pages 451-469

Files

Share

How to cite

Statistics