Fuzzy information-theoretic feature selection via relevance, redundancy, and complementarity criteria

The concepts of feature relevance, redundancy, and complementarity are very important for identifying optimal feature subsets and designing effective feature evaluation criteria when developing information-theoretic feature selection methods. However, the aforementioned concepts are generally defined based on classical information-theoretic measures that work with discrete features. Therefore, it is very difficult to directly apply them to handle continuous features. In this paper, we investigate the concepts of feature relevance, redundancy, and complementarity based on fuzzy information-theoretic measures to address continuous features. More specifically, we examine some fuzzy information-theoretic measures for any finite number of fuzzy T -equivalence relations. From a conceptual point of view, we present theoretical definitions of feature relevance, redundancy, and complementarity by using these fuzzy information-theoretic measures. According to these theoretical definitions, we introduce a computationally effective feature evaluation criterion that employs a weighting scheme to combine feature relevance, redundancy, and complementarity. We propose a feature selection algorithm by combining the feature evaluation criterion with the sequential forward search strategy. To verify the effectiveness of our method, we compare it with some state-of-the-art feature selection methods through extensive experiments. The experimental results show that our method achieves better performance.