Improvement of Min-Entropy Evaluation Based on Pruning and Quantized Deep Neural Network

In the field of information security, the unpredictability of random numbers plays determinant role according to the security of cryptographic systems. However, limited by the capability of pattern recognition and data mining, statistical-based methods for random number security assessment can only detect whether there are obvious statistical flaws in random sequences. In recent years, some machine learning-based techniques such as deep neural networks and prediction-based methods applied to random number security have exhibited superior performance. Concurrently, the proposed deep learning models bring out issues of large number of parameters, high storage space occupation and complex computation. In this paper, for the challenge of random number security analysis: building high-performance predictive models, we propose an effective analysis method based on pruning and quantized deep neural network. Firstly, we train a temporal pattern attention-based long short-term memory (TPA-LSTM) model with complex structure and good prediction performance. Secondly, through pruning and quantization operations, the complexity and storage space occupation of the TPA-LSTM model were reduced. Finally, we retrain the network to find the best model and evaluate the effectiveness of this method using various simulated data sets with known min-entropy values. By comparing with related work, the TPA-LSTM model provides more accurate estimates: the relative error is less than 0.43%. In addition, the model weight parameters are reduced by more than 98% and quantized to 2 bits (compression over 175x) without accuracy loss.