RAT-LLM: A Retrieval-Augmented Temporal LLM for Zero-Shot Soft Sensing in Multimode Industrial Processes

The inherent multimode characteristics of industrial processes often lead to frequent and severe distribution shifts in data. Such shifts significantly hinder the generalization of data-driven soft sensors, particularly when unknown operating modes are encountered. To address this challenge, this paper proposes the RAT-LLM, a Retrieval-Augmented Temporal Large Language Model for zero-shot soft sensing (ZSSS). The RAT-LLM leverages a pre-trained Large Language Model (LLM) as its backbone to generate quality predictions guided by retrieved segments. Specifically, a Retrieval Module is first developed to identify dynamically similar candidate segments. This module comprises a multi-scale retrieval encoder, a retrieval knowledge base, and a retriever, that projects raw process segments into a latent space for embedding-based retrieval. A Multi-grained Temporal Representation Module (MTRM) is then designed to capture fine-grained and coarse-grained dependencies from input and retrieved sequences. The multi-grained representations are input into the LLM. To bridge the modality gap between continuous industrial signals and the LLM’s linguistic latent space, the model employs Parameter-Efficient Fine-Tuning (PEFT) to fine-tune the pre-trained LLM backbone. Furthermore, a Similarity-Utility Alignment Mechanism is introduced to align retrieval similarity metrics with downstream predictive accuracy. Case studies on two real-world industrial processes demonstrate that the proposed RAT-LLM achieves strong zero-shot generalization, outperforming state-of-the-art baselines on previously unseen operating modes. Note to Practitioners—In modern industrial processes, a persistent challenge is that data-driven soft sensors fail when a new operating mode is encountered. Traditionally, such failures force engineers to either manually recalibrate models or collect vast amounts of new labeled data to retrain them, both of which are costly and time-consuming. To address this challenge, this paper proposes the RAT-LLM, a model that enables zero-shot soft sensing. It can provide accurate quality predictions for new operating modes without requiring any retraining on those specific modes. Instead of learning from scratch, RAT-LLM automatically searches a knowledge base for the most relevant historical operating data and uses an adapted Large Language Model to infer quality values guided by these past patterns. For plant operators, this approach means drastically reduced model maintenance effort and highly reliable real-time monitoring, even when production conditions are constantly changing.