CACO: Chiplet-based FHE Acceleration with Inter-Chiplet Communication Optimizations

Fully Homomorphic Encryption (FHE) is a break-through technology for privacy-preserving computation, enabling secure computation directly on encrypted data. However, its practical adoption is limited by substantial performance overhead. Numerous hardware accelerators have been proposed to address this challenge, but they typically require large circuit areas, resulting in low manufacturing yield. To improve yield, recent efforts have explored chiplet-based architectures for FHE acceleration. Nevertheless, existing chiplet designs remain constrained by die-to-die (D2D) communication bottlenecks, hindering further performance scaling. In this work, we propose CACO, a chiplet-based FHE accelerator that minimizes D2D communication by exploiting higher-level parallelism in FHE. Through analysis of existing chiplet designs, we make two key observations: 1) clustering FHE operations within fewer chiplets reduces D2D communication; 2) exploiting higher-level parallelism further improves performance. Building on these insights, we identify four challenges in enabling such mappings: 1) the tradeoff between D2D communication and available D2D links when constraining FHE operations; 2) limited parallelism in ASIC-oriented bootstrapping algorithms; 3) additional D2D communication introduced by switching between different compute patterns; and 4) increased on-chip memory demands caused by parallel execution of FHE operations. To address these challenges, we first analyze D2D communication under different mapping strategies. We then restructure the bootstrapping algorithm to expose higher-level parallelism by breaking inter-operation dependencies. Next, we introduce a mapping optimization that prevents unnecessary inter-chiplet transfers by broadcasting intermediate results in advance to the destination chiplet. Finally, we propose a memory management strategy to reduce on-chip memory requirements. Experimental results demonstrate that CACO achieves a 1.50× speedup over the state-of-the-art chiplet-based design.