Research on Generative Pre-Trained Model Evaluation Based on Causality Analysis

At present, generative models have developed to produce creative sentences and widely used in existing applications. Specifically, large language pre-trained models may produce various generative results from different inputs, which can assist users to finish their targets. However, these generative results may obtain unacceptable information caused by illegal and uncontrolled sentences inputs. Inspired by the explainable mechanism for machine learnings, we utilize the causality analysis to investigate the relationships between the generative results and corresponding inputs. Initially, we utilize a sentiment text dataset to pre-trained model and obtain the generative analysis results. Subsequently, we utilize the explainable model to select the words in the dataset, which means these words can significantly influence the generative results of the pre-trained model. Finally, we test the selected words and corresponding sentiment analysis results with traditional classification models and non-selected texts. From our experimental simulations, we can observe that the generative model is sensitive to some certain words, which can significantly improve the sentiment analysis accuracy when only process these sensitive words.