A Hybrid Processing-in-Memory and Computing-in-Memory Architecture for Large Language Model Inference in Edge Devices

Deploying Large Language Models (LLMs) on resource-constrained edge devices is critically challenged by the "memory wall" bottleneck, where energy-intensive data movement between processors and memory dominates system costs. This paper introduces a hybrid Processing-in-Memory (PIM) and Computing-in-Memory (CIM) architecture that synergistically integrates DRAM-based PIM and SRAM-CIM to accelerate Transformer inference. Our key innovation lies in strategically partitioning the attention computation between the two technologies: SRAM-CIM efficiently handles the dynamic Value (V) cache operations prone to access conflicts in DRAM-PIM, while DRAM-PIM accelerates other linear layers. To maximize efficiency, we co-design a hierarchical channel-bank data mapping scheme and an asymmetry-aware workload assignment. Implemented and verified in 28nm technology, our architecture achieves a 1.51 times speedup and 1.24 times reduction in energy consumption compared to DRAM-PIM-only baselines in the focused Transformer layers for Llama2-7B inference, demonstrating potential for edge deployment.