嵌入
计算机科学
歪斜
残余物
算法
物理系统
变压器
忠诚
偏微分方程
人工神经网络
背景(考古学)
Lift(数据挖掘)
反演(地质)
编码器
人工智能
离散化
反向传播
编码(内存)
数学
反问题
高保真
因果关系(物理学)
上下文模型
压缩性
微分方程
作者
Ehsan Zeraatkar,Rodion M. Podorozhny,Jelena Tešić
标识
DOI:10.48550/arxiv.2603.27929
摘要
Reconstructing continuous physical fields from sparse, irregular observations is a central challenge in scientific machine learning, particularly for systems governed by partial differential equations (PDEs). Existing physics-informed methods typically enforce governing equations as soft penalty terms during optimization, often leading to gradient imbalance, instability, and degraded physical consistency under limited data. We introduce the Physics-Guided Transformer (PGT), a neural architecture that embeds physical structure directly into the self-attention mechanism. Specifically, PGT incorporates a heat-kernel-derived additive bias into attention logits, encoding diffusion dynamics and temporal causality within the representation. Query coordinates attend to these physics-conditioned context tokens, and the resulting features are decoded using a FiLM-modulated sinusoidal implicit network that adaptively controls spectral response. We evaluate PGT on the one-dimensional heat equation and two-dimensional incompressible Navier-Stokes systems. In sparse 1D reconstruction with 100 observations, PGT achieves a relative L2 error of 5.9e-3, significantly outperforming both PINNs and sinusoidal representations. In the 2D cylinder wake problem, PGT uniquely achieves both low PDE residual (8.3e-4) and competitive relative error (0.034), outperforming methods that optimize only one objective. These results demonstrate that embedding physics within attention improves stability, generalization, and physical fidelity under data-scarce conditions.
科研通智能强力驱动
Strongly Powered by AbleSci AI