LUT Tensor Core: Lookup Table Enables Efficient Low-Bit LLM Inference Acceleration

LUT T ENSOR C ORE is introduced, a software-hardware co-design optimized for low-bit LLM inference that introduces software-based operator fusion and table symmetrization techniques to optimize table precompute and table storage and designs an end-to-end compilation stack with new instructions for LUT-based mpGEMM, enabling efficient LLM compilation and optimizations.