InDenT: An Energy-Efficient 2.5D System With Heterogeneous Reconfigurable Interconnect for Dense Foundational Models

Dense transformer algorithms have shown supreme accuracy while executing machine intelligence tasks by introducing dense connections between various computing blocks. However, the introduction of dense connections increases computation and communication volume which increases latency and energy consumption. Therefore, we propose a 2.5D system with heterogeneous reconfigurable interconnect for energy-efficient execution of dense transformer algorithms. Our proposed 2.5D system includes in-memory computing (IMC) as well as CMOS components to execute different types of computations involved in a dense transformer. Our proposed heterogeneous reconfigurable interconnect comprises on-package wired as well as wireless links to reduce communication latency. The interconnect is reconfigurable since at runtime it is decided which wireless links are to be utilized. Extensive experimental evaluations with different dense transformers show an improvement of up to 39.32% in communication latency with respect to state-of-the-art 2.5D systems.