Benchmarking validity indices for evolutionary K-means clustering performance

Abstract K-Means is a well-established clustering algorithm widely used in data analysis and various real-world applications. However, its requirement for a predefined number of clusters limits its effectiveness in automatic clustering tasks. To address this, metaheuristic optimisation algorithms have been integrated into K-Means, leading to the development of Evolutionary K-Means clustering approaches. These methods often rely on internal validity indices as fitness functions to automatically determine both the optimal number of clusters and the clustering configuration. However, the effectiveness of internal validity indices is often data-dependent, as most are tailored to specific data characteristics. Consequently, the choice of validity index can significantly influence clustering outcomes. This study evaluates the performance of fifteen internal validity indices within the Enhanced Firefly Algorithm-K-Means (FA-K-Means) framework, an evolutionary approach that integrates Firefly metaheuristics with the classical K-Means algorithm. The performance of each index is assessed across a diverse collection of real-life and synthetic datasets with varying structures. The results reveal that the Calinski-Harabasz (CH) and Silhouette indices consistently outperform others, offering more reliable clustering performance. These findings provide practical guidance for selecting appropriate fitness functions in Evolutionary K-Means algorithms for automatic clustering tasks.