Novel Parallel Digital Optical Computing System (DOC) for Generative A.I

Generative A.I.'s (GAI) popularity has made photonics-based computation an attractive approach for its potential to meet the demands for higher energy efficiency performance (EEP). However, previous optical solutions for multiply-accumulate (MAC) operations focused on either analog architecture [1]–[7] which is limited by its precision and data conversion, or free-space optical architecture with limited scalability [8]. Here, a world's first on-chip large-scale Digital Optical Computing System (DOC) for GAI training is reported. DOC employs a novel wafer-based system integration technology featuring multilayer low-loss photonic interconnect fan-out (PIFO) and EIC/PIC stack architecture leveraging TSMC SoIC®. It reduces the data movement and memory hierarchy leading to improvement in critical path latency and system energy efficiency (EE). Compared to conventional electrical designs, DOC can be scaled to larger coherent networks and operate at higher speeds with lower energies per MAC operation. A low energy consumption of <0.08 pJ/MAC at 8-bit operation with a >20x improvement in EEP compared to the state-of-the-art GPU [10] is achieved for a 512 x 512 MAC large scale operation. The EEP further improves at higher precisions due to the relative minimal fan-out energy. This architecture has full potential for continuous EEP scaling in future generations.