SASDenSebLE: A Compact Vision Transformer Inference Architecture With Saturation-Approximate Softmax Dataflow Enabling Sequence-Parallelism Boosted Layer-Fusion Execution

Transformer-based neural networks (NNs) have been dominant in modern AI due to their superior accuracy and generalization over traditional models. However, the quadratic complexity and dataflow challenges of self-attention hinder their efficient deployment on edge devices, especially for latency-sensitive, single-batch inference. Limited data parallelism necessitates both pipeline and tensor parallelism. Specifically, sequence parallelism offers strong scaling potential in domain-specific accelerators (DSAs), but is constrained by the temporal dependencies involved in the max-finding operations of softmax. This work presents SASDenSebLE, a compact, scalable vision Transformer (ViT) inference architecture co-designed with a max-finding-free softmax approximation across micro-architecture, architecture, and algorithm levels. A dense-sparse-hybrid datapath transforms large-value softmax operations into linear projections, enabling layer-fusion. A hierarchical system further improves sequence parallelism without excessive on-chip buffering or off-chip traffic. Experiments show up to $2.83\times$ , $28.02\times$ , and $1.40\times$ speedup over baseline DSAs and a state-of-the-art ViT accelerator design, respectively, with negligible accuracy loss. Demonstration on the camera encoder of a recent bird’s-eye-view model shows up to $10.82\times$ speedup over a typical edge GPU.