Application Defined On-chip Networks for Heterogeneous Chiplets: An Implementation Perspective

With the help of advanced packaging technologies to integrate multiple chips (e.g., CPU, AI, IO), a chiplet-based SoC design process can enable fast system construction. However, the design of network-on-chip used within the individual chiplets and across chiplets is an extremly challenge. We introduce the design process and methodology of a bufferless multi-ring NoC for heterogeneous chiplet-based SoC. Our design is portable and can be used in diverse scenarios, like Server-CPU, AI-Processor, and Baseband-Processor.The co-design of the application, architecture, and implementation is the key to make the system power efficient and high performance. We determined many architectural design choices by reflecting an analysis of a set of target applications by application teams and several physical implementation constraints provided by development teams. In this paper, we present the pragmatic practice of our co-design effort for the NoC. As a result, the system has been proven to achieve 16TB/s bandwidth in an AI processor and low latency, in a server CPU with nearly one hundred cores.