SLM-Arch: Trade-Off Analysis and Optimized Hardware Design for Small Language Models

Small language models (SLMs) are increasingly deployed in resource-constrained environments, including edge devices and domain-specific applications. To accelerate SLM inference, recent research has focused on developing specialized accelerators to improve execution efficiency. However, designing efficient hardware accelerators for SLMs remains a significant challenge due to the different requirements of applications (e.g., chip area, power) and the complex interplay between architecture parameters and workload characteristics. This paper presents a design space exploration (DSE) framework of SLM accelerators. We derive empirical formulas for the optimized processing element (PE) array size and establish an analytical method for memory capacities to minimize redundant memory accesses. In addition, we also explore the optimized combination of PE array size, memory capacity, bandwidth, and dataflow. Furthermore, we investigate area-constrained scenarios (2 mm2, 7 mm2, 30 mm2) and provide guidelines for allocating computation and memory resources under realistic deployment conditions. Experiment results show that the optimized PE array size is around 240 × 240 for input stationary (IS) and output stationary (OS), while weight stationary (WS) requires at least 300 × 300 but with lower memory resource demands. With relaxed area constraints, at least 30 GB/s of memory bandwidth is needed for area full utilization, and area should be allocated to both adequate memory capacity and sufficient computation resources. Under strict area limits, however, maximizing the PE array is most effective. Overall, our framework offers practical insights into hardware parameter selection, enabling optimized performance deployment of SLM accelerators across diverse application domains.