Tell me more: integrating LLMs in a cultural heritage website for advanced information exploration support

Abstract Cultural Heritage websites’ capability to satisfy diverse information needs is limited by their high-quality but constrained knowledge bases. Thus, we investigate their extension with external large language models (LLMs), enriching the provision of cultural content by leveraging LLMs’ continuous collection and integration of information from heterogeneous data sources. This extension raises important challenges in synchronizing the LLM’s behavior with the user’s browsing activity on the website to offer a unified interaction environment. To address these challenges, we propose a loosely coupled integration model that provides users with curated content and an assisted question-answering function to answer information needs that the system’s knowledge base fails to cover. Our model is agnostic to the LLM and synchronizes its behavior with the user’s browsing activity through implicit prompt engineering. We tested a baseline website without LLM integration, one with free-text interaction with the LLM, and another that combines free-text interaction with the suggestion of context-dependent questions. In a user study involving 44 participants, we found that the LLM-powered website has higher usability and that context-dependent question suggestions further enhance user experience, especially for people with low curiosity levels (according to Curiosity and Exploration Inventory-II - CEI-II) who are guided in formulating effective questions. This shows the potential of LLMs to enrich engagement with existing Cultural Heritage websites.