计算机科学
范畴变量
余弦相似度
人工智能
雅卡索引
相似性(几何)
数据挖掘
机器学习
聚类分析
图像(数学)
作者
Dan Ruan,Weber Shao,Julius P. Wong,Darlene Veruttipong,Michael Steinberg,Daniel A. Low,Patrick A. Kupelian
出处
期刊:Medical Physics
[Wiley]
日期:2016-06-01
卷期号:43 (6Part14): 3485-3485
被引量:4
摘要
Purpose: growing partnership and contributors to registry efforts an efficient way to perform data integration, de‐duplication, and cleansing, amongst other tasks. This project aims to develop an automatic curation module to discover, clean, and transform new data, and semantically integrate them with the composite hosting registry upon de‐duplication. Methods: We have investigated various similarity metrics for various attributes and value types in the registry. More specifically, cosine word similarity, cosine term frequency‐inverse document frequency, cosine trigram similarity are used to measure similarity between free text fields; Jaccard similarity and histogram matching are used for categorical values, Naive Bayesian probability in combination with Welch's t‐test values are used to measure likelihood of matching for numerical fields. A supervised learning framework is then adopted where these similarity values are used as input to regress out a decision of amongst {“automatic aggregation with high confidence”, “no match (hence expand current registry)”, and “likely match upon confirmation”}. Multi‐class logistic regression, and classification and regression tree (CART) techniques were used. Seven previous rolling versions of the registry with variations in terminology and data duration were used in a cross‐validation setting to test the applicability of the proposed approach. Results: Initial results showed that automatic curation can properly label more than 80% of the label and data fields, significantly alleviating the burden of human operator and the risk of human error. The major limitation of the current approach is ambiguity, e.g., an input tag “MRI” yields multiple potential match. We are investigating the incorporation of context knowledge to reduce such ambiguity to facilitate both automatic curation and human confirmation. Conclusion: (Semi‐)automatic curation provides a sustainable pathway to integrate data both longitudinally and across multiple institutions and data sources. Our preliminary effort has demonstrated the feasibility and potential efficacy of utilizing machine learning techniques for such purposes. This work is supported in part by a Varian research grant.
科研通智能强力驱动
Strongly Powered by AbleSci AI