计算机科学
人工智能
机器学习
二元分类
深度学习
任务(项目管理)
选择(遗传算法)
班级(哲学)
监督学习
人工神经网络
支持向量机
水准点(测量)
工程类
大地测量学
系统工程
地理
作者
Luís Ferreira,André Pilastri,Carlos Manuel Martins,Pedro Pires,Paulo Cortez
出处
期刊:International Joint Conference on Neural Network
日期:2021-07-18
被引量:40
标识
DOI:10.1109/ijcnn52387.2021.9534091
摘要
This paper presents a benchmark of supervised Automated Machine Learning (AutoML) tools. Firstly, we analyze the characteristics of eight recent open-source AutoML tools (Auto-Keras, Auto-PyTorch, Auto-Sklearn, AutoGluon, H2O AutoML, rminer, TPOT and TransmogrifAI) and describe twelve popular OpenML datasets that were used in the benchmark (divided into regression, binary and multi-class classification tasks). Then, we perform a comparison study with hundreds of computational experiments based on three scenarios: General Machine Learning (GML), Deep Learning (DL) and XGBoost (XGB). To select the best tool, we used a lexicographic approach, considering first the average prediction score for each task and then the computational effort. The best predictive results were achieved for GML, which were further compared with the best OpenML public results. Overall, the best GML AutoML tools obtained competitive results, outperforming the best OpenML models in five datasets. These results confirm the potential of the general-purpose AutoML tools to fully automate the Machine Learning (ML) algorithm selection and tuning.
科研通智能强力驱动
Strongly Powered by AbleSci AI