计算机科学
成对比较
多样性(控制论)
情报检索
多样性(政治)
文档聚类
语料库
文件结构说明
基质(化学分析)
自然语言处理
人工智能
数据科学
万维网
聚类分析
复合材料
材料科学
人类学
社会学
XML
作者
Kevin Bache,David J. Newman,Padhraic Smyth
出处
期刊:
[Association for Computing Machinery]
日期:2013-08-11
卷期号:: 23-31
被引量:44
标识
DOI:10.1145/2487575.2487672
摘要
Quantitative notions of diversity have been explored across a variety of disciplines ranging from conservation biology to economics. However, there has been relatively little work on measuring the diversity of text documents via their content. In this paper we present a text-based framework for quantifying how diverse a document is in terms of its content. The proposed approach learns a topic model over a corpus of documents, and computes a distance matrix between pairs of topics using measures such as topic co-occurrence. These pairwise distance measures are then combined with the distribution of topics within a document to estimate each document's diversity relative to the rest of the corpus. The method provides several advantages over existing methods. It is fully data-driven, requiring only the text from a corpus of documents as input, it produces human-readable explanations, and it can be generalized to score diversity of other entities such as authors, academic departments, or journals. We describe experimental results on several large data sets which suggest that the approach is effective and accurate in quantifying how diverse a document is relative to other documents in a corpus.
科研通智能强力驱动
Strongly Powered by AbleSci AI