弦(物理)
范畴变量
计算机科学
基数(数据建模)
可扩展性
散列函数
理论计算机科学
可解释性
人工智能
数据挖掘
机器学习
数学
程序设计语言
数据库
数学物理
作者
Patricio Cerda,Gael P. Varoquaux
标识
DOI:10.1109/tkde.2020.2992529
摘要
Statistical models usually require vector representations of categorical variables, using for instanceone-hot encoding. This strategy breaks down when the number of categories grows, as it creates high-dimensional feature vectors. Additionally, for string entries, one-hot encoding does not capture morphological information in their representation. Here, we seek low-dimensional encoding of high-cardinality string categorical variables. Ideally, these should be: scalable to many categories; interpretable to end users; and facilitate statistical analysis. We introduce two encoding approaches for string categories: aGamma-Poisson matrix factorizationon substring counts, and amin-hash encoder, for fast approximation of string similarities. We show that min-hash turns set inclusions into inequality relations that are easier to learn. Both approaches are scalable and streamable. Experiments on real and simulated data show that these methods improve supervised learning with high-cardinality categorical variables. We recommend the following: if scalability is central, the min-hash encoder is the best option as it does not require any data fit; if interpretability is important, the Gamma-Poisson factorization is the best alternative, as it can be interpreted as one-hot encoding on inferred categories with informative feature names. Both models enable autoML on string entries as they remove the need for feature engineering or data cleaning.
科研通智能强力驱动
Strongly Powered by AbleSci AI