一般化
归纳偏置
核(代数)
人工智能
计算机科学
人工神经网络
简单(哲学)
核方法
机器学习
算法
泛化误差
回归
深度学习
模式识别(心理学)
深层神经网络
数学
统计学习理论
分布的核嵌入
训练集
曲线拟合
统计学习
应用数学
均方预测误差
作者
Abdulkadir Canatar,Blake Bordelon,Cengiz Pehlevan
标识
DOI:10.1038/s41467-021-23103-1
摘要
A theoretical understanding of generalization remains an open problem for many machine learning models, including deep networks where overparameterization leads to better performance, contradicting the conventional wisdom from classical statistics. Here, we investigate generalization error for kernel regression, which, besides being a popular machine learning method, also describes certain infinitely overparameterized neural networks. We use techniques from statistical mechanics to derive an analytical expression for generalization error applicable to any kernel and data distribution. We present applications of our theory to real and synthetic datasets, and for many kernels including those that arise from training deep networks in the infinite-width limit. We elucidate an inductive bias of kernel regression to explain data with simple functions, characterize whether a kernel is compatible with a learning task, and show that more data may impair generalization when noisy or not expressible by the kernel, leading to non-monotonic learning curves with possibly many peaks.
科研通智能强力驱动
Strongly Powered by AbleSci AI