Subkv: Quantizing Long Context KV Cache for Sub‐Billion Parameter Language Models on Edge Devices

Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks. However, their substantial computational and memory requirements present significant challenges for widespread deployment on edge devices.