IEAgent: A Zero-Shot Information Extraction Agent with Knowledge RAG for Unstructured Data

Recent advancements in large language models (LLMs) have shown their ability to handle new tasks with minimal effort, often by relying solely on natural language prompts. This opens up the possibility of performing zero-shot information extraction (IE) without parameter tuning or reliance on large labeled datasets. However, IE inherently involves a complex pipeline that includes semantic analysis, entity recognition, relation extraction, and the integration of external domain knowledge. Despite of this, the existing approaches have not fully addressed the effective planning and coordination of these stages. In this paper, we present IEAgent¹, an agent-based framework designed to overcome these challenges. IEAgent not only uses LLMs as core reasoning engines but also employs agent-like mechanisms to autonomously plan, execute, and refine the entire IE process. It incorporates a suite of external tools to access real-time and domain-specific knowledge, which the agent orchestrates through structured prompts and adaptive decision-making steps. Experiments show that our method outperforms SUMASK on FewRel and Wiki-ZLS datasets by 7.46% and 0.45% in F1 score, respectively, and exceeds ChatIE on NYT11-HRL and DUIE2.0 with F1 gains of 16.35% and 10.92%.