HGNN-T5 PEGASUS: A Hybrid Approach for Chinese Long Text Summarization

Currently, the performance of abstractive summarization is generally superior to extractive summarization, but it is inhibited for the long text by the complexity. Extractive summarization, though high speed, is redundancy and lack of sentences coherence. Based on the above, for Chinese long text summarization, we propose a hybrid model HGNN-T5 PEGASUS which includes two stages. In the first stage, we use a heterogeneous graph-based extractive model to obtain a shorter extractive summarization than the original text. The heterogeneous graph-based neural network (HGNN) incorporates sentence-level and word-level semantic nodes, which can enrich the relationship between sentences. In the second stage, we choose the T5 PEGASUS as our base model and add a category prediction mechanism to alleviate the Out-Of-Vocabulary problem and improve fidelity. Moreover, a more simplified sparse softmax is introduced to T5 PEGASUS to avoid overfitting. To demonstrate the effectiveness of our model, we constructed the SCHOLAT text summarization dataset. The results of our experiments show that the proposed model outperforms other baseline models on both the NLPCC 2018 and SCHOLAT datasets.