Steering Artificial Intelligence Models Toward Comprehensive Domain Knowledge

作者
Anindya Das Antar
标识
DOI:10.7302/28235
摘要

Model engineers build decision‑support systems using Artificial Intelligence (AI) models, expecting that these models will achieve, if not exceed, human expert‑level performance in decision‑making. To meet this goal, AI models must make accurate decisions, explain reasoning, and ground outputs in comprehensive domain knowledge. This knowledge encompasses all relevant and meaningful information synthesized by experts within the domains these models represent or operate in. However, AI models often lack such knowledge, either because it is missing from training data or algorithms fail to learn correct representations. This dissertation refers to this missing knowledge as "latent domain knowledge"---information known to experts but latent from the model's perspective. While many stakeholders possess domain knowledge, this dissertation focuses on model engineers and domain experts, who are most directly involved in developing, evaluating, and interacting with AI models throughout their lifecycle. To address latent knowledge, this dissertation first shows that integrating domain knowledge through collaboration with domain experts significantly improves model performance, outperforming baselines trained solely on expert-provided data. It then shows that users can identify latent knowledge by interactively exploring models through domain-relevant "what-if" scenarios. Next, it proposes a Bayesian inference–based method to probabilistically evaluate and select effective steering mechanisms (e.g., natural language prompt-based guardrails for Large Language Models (LLMs)) that align model outputs with expert users' knowledge. Although domain experts may lack AI expertise, they value autonomy and may not want to rely on model engineers for post-deployment monitoring and debugging. To support this, the dissertation introduces interactive steering tools for models that do not accept prompt-based guardrails, enabling users to steer the model's input–output relationships directly. The Bayesian method validates such steering in the background and retains those that best align outputs with empirical data. These exploration and steering tools are integrated in a data-, domain-, and model-agnostic toolbox named Visual Interactive Model Explorer (VIME). VIME enables users to investigate and steer model behavior toward desired outputs and knowledge. This dissertation details iterative refinements of VIME based on user needs identified through prior literature and formative studies. It presents simplified user evaluation findings on how effectively model engineers and domain experts can identify and introduce latent knowledge, and informs the broader adoption of domain knowledge–aligned AI-based decision-support systems.

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
斯文败类的应助被夏诗婷采纳,获得10
刚刚
否决权完成签到,获得积分10
刚刚
DW的应助被顺心的星月采纳,获得10
刚刚
大白兔完成签到 ,获得积分10
刚刚
秋秋发布了新的文献求助10
1秒前
爱火未灭完成签到,获得积分10
1秒前
所所的应助被哈哈采纳,获得10
1秒前
Emily完成签到,获得积分20
2秒前
YaoHui完成签到,获得积分10
4秒前
4秒前
莱特昊完成签到,获得积分10
4秒前
4秒前
这么近那么远完成签到,获得积分20
5秒前
丁丁当当发布了新的文献求助10
5秒前
Yurmas完成签到,获得积分10
6秒前
啦你完成签到 ,获得积分10
7秒前
7秒前
8秒前
molihuakai的应助被yz123采纳,获得10
8秒前
吉吉国王完成签到 ,获得积分10
8秒前
YaoHui发布了新的文献求助10
9秒前
大模型的应助被小燕尾条采纳,获得10
10秒前
zhu发布了新的文献求助10
10秒前
情怀的应助被咎如天采纳,获得10
10秒前
molihuakai的应助被HJH采纳,获得10
10秒前
11秒前
11秒前
11秒前
啦啦啦发布了新的文献求助10
12秒前
LIGE发布了新的文献求助10
13秒前
13秒前
虎虎发布了新的文献求助10
14秒前
Dht完成签到,获得积分10
15秒前
的法国队发布了新的文献求助10
16秒前
wsx4321发布了新的文献求助10
17秒前
18秒前
Lee发布了新的文献求助10
18秒前
酷波er的应助被kuba采纳,获得10
18秒前
18秒前
yu完成签到,获得积分20
19秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
John McDowells Bildungstheorie 800
八旗制度の研究 600
plantTFDB 508
Resource Recovery Handbook 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8175142
求助须知:如何正确求助?哪些是违规求助? 9566928
关于积分的说明 21789123
捐赠科研通 7769522
什么是DOI,文献DOI怎么找? 3431822
关于科研通互助平台的介绍 2556078
邀请新用户注册赠送积分活动 2477222