Enabling Efficient and Accurate Privacy-Preserving Image-Text Retrieval in Public Cloud

The growth of public cloud services has raised privacy and security concerns for outsourced data. Privacy-preserving image-text retrieval aims to protect outsourced data while enabling cross-modal retrieval between images and texts. However, existing methods mainly focus on single-modal search and face challenges with high computation overhead and low accuracy. In this paper, we propose EmpowerIR, a novel privacy-preserving image-text retrieval framework that enables efficient and accurate searches in multi-user settings. We first utilize the Vision Language Foundation Model CLIP to generate embeddings for each image and text. Following this, a privacy concealment principal component outsourcing method is designed, which blinds the results of principal component analysis using private keys. Then, we develop a key conversion protocol and distribute switch keys to the cloud, enabling accurate image-text retrieval in the multi-user settings. Extensive experiments on real-world datasets show that EmpowerIR outperforms existing methods in terms of search accuracy and efficiency.