Learning Robust to Distributional Uncertainties and Adversarial Data

对抗制 计量经济学 计算机科学 统计 数学 人工智能
作者
Alireza Sadeghi,Gang Wang,Georgios B. Giannakis
出处
期刊:IEEE journal on selected areas in information theory [Institute of Electrical and Electronics Engineers]
卷期号:5: 105-122 被引量:4
标识
DOI:10.1109/jsait.2024.3381869
摘要

Successful training of data-intensive deep neural networks critically rely on vast, clean, and high-quality datasets. In practice however, their reliability diminishes, particularly with noisy, outlier-corrupted data samples encountered in testing. This challenge intensifies when dealing with anonymized, heterogeneous data sets stored across geographically distinct locations due to e.g., privacy concerns. This present paper introduces robust learning frameworks tailored for centralized and federated learning scenarios. Our goal is to fortify model resilience with a focus that lies in (i) addressing distribution shifts from training to inference time; and, (ii) ensuring test-time robustness, when a trained model may encounter outliers or adversarially contaminated test data samples. To this aim, we start with a centralized setting where the true data distribution is considered unknown, but residing within a Wasserstein ball centered at the empirical distribution. We obtain robust models by minimizing the worst-case expected loss within this ball, yielding an intractable infinite-dimensional optimization problem. Upon leverage the strong duality condition, we arrive at a tractable surrogate learning problem. We develop two stochastic primal-dual algorithms to solve the resultant problem: one for -accurate convex sub-problems and another for a single gradient ascent step. We further develop a distributionally robust federated learning framework to learn robust model using heterogeneous data sets stored at distinct locations by solving per-learner’s sub-problems locally, offering robustness with modest computational overhead and considering data distribution. Numerical tests corroborate merits of our training algorithms against distributional uncertainties and adversarially corrupted test data samples.

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
烹全鱼宴发布了新的文献求助10
刚刚
Nuyoah完成签到,获得积分10
刚刚
1秒前
笑点低不愁完成签到,获得积分20
1秒前
qyd完成签到,获得积分20
1秒前
丘比特的应助被裕溪采纳,获得10
1秒前
2秒前
AKTAO完成签到,获得积分10
2秒前
阿白发布了新的文献求助10
2秒前
活力板凳完成签到,获得积分10
3秒前
在水一方的应助被XIYBO采纳,获得10
3秒前
我是老大的应助被寒冷鸭子采纳,获得10
3秒前
4秒前
甜美的尔柳完成签到,获得积分10
4秒前
4秒前
长风完成签到,获得积分10
4秒前
Ava的应助被xie采纳,获得10
5秒前
pandaheld完成签到,获得积分10
5秒前
Chelsea发布了新的文献求助20
6秒前
6秒前
6秒前
superchen完成签到,获得积分10
6秒前
科研通AI6.3的应助被爱笑丹云采纳,获得10
7秒前
斯文败类的应助被小杨采纳,获得10
7秒前
7秒前
hhh完成签到,获得积分10
8秒前
8秒前
爱科研的小民完成签到,获得积分10
8秒前
在望的应助被科小白采纳,获得10
8秒前
Owen的应助被务实的忆雪采纳,获得10
8秒前
风吹麦浪发布了新的文献求助10
9秒前
9秒前
风吹麦浪发布了新的文献求助10
9秒前
9秒前
wang5945发布了新的文献求助10
9秒前
10秒前
wwj完成签到,获得积分20
10秒前
howay发布了新的文献求助10
10秒前
11秒前
研友_VZG7GZ的应助被AKTAO采纳,获得10
11秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
八旗制度の研究 600
Boundaries and Bodhisattvas 500
高分子基功能界面水蒸发材料与应用研究 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8181279
求助须知:如何正确求助?哪些是违规求助? 9571198
关于积分的说明 21817073
捐赠科研通 7777612
什么是DOI,文献DOI怎么找? 3433454
关于科研通互助平台的介绍 2557692
邀请新用户注册赠送积分活动 2479519