Dynamic DNN Nodes Drop-out Rate Analysis

The Dropout method is very useful when dealing with the overfitting problem that occurs during the training process of a data set whose data size is in the range thousands to tens of thousands. But the dropout rate in the current dropout method is fixed. This work aims to propose several methods that can dynamically adjust the dropout rate during training. Compared with the traditional dropout method, the method of dynamically adjusting the dropout rate can more effectively prevent the overfitting phenomenon, and ensure that the data is fully trained and improve the accuracy. The method can improve dropout performance and plays a more critical role in the latest federated learning and other fields.