Effects of the Number of Hyperparameters on the Performance of GA-CNN

The performance of a machine learning algorithm is highly dependent on its hyperparameters. However, hyperparameter optimization is not a trivial task as it is problem-specific. The difficulty rises when coupled with a larger number of hyperparameters resulting in high search space dimensions. The common understanding seems to be that the optimization is only done on limited hyperparameters. Indeed, a larger number of hyperparameters have not been commonly utilized in hyperparameter optimization. This study investigates the role of hyperparameters by using a genetic algorithm (GA) as the main optimization method for a convolutional neural network (CNN). The novelty of this study is two-fold. Firstly, we defined 20 hyperparameters and their ranges, specifically for text classification. Secondly, we conducted experiments with different numbers of hyperparameters and different numbers of optimized hyperparameters. GA-CNN was evaluated using a disaster tweets dataset and compared to the other methods, i.e., grid search, random search, TPE, TPOT, CNN, LSTM, CNN-LSTM, and BERT. The experimental results demonstrated that the proposed method shows better performance over other methods. The results also showed that a larger number of hyperparameters and layer-specific hyperparameter values are indeed important.