When Not to Trust Language Models: Investigating Effectiveness and Limitations of Parametric and Non-Parametric Memories
When Not to Trust Language Models: Investigating Effectiveness and Limitations of Parametric and Non-Parametric Memories
Alex Troy Mallen,Akari Asai,3 作者,Daniel Khashabi
2022 · DOI: 10.48550/arXiv.2212.10511
arXiv.org · 引用 152 次
TLDR
It is shown that LMs struggle with less popular factual knowledge, and that scaling fails to appreciably improve memorization of factual knowledge in the tail, and that retrieval-augmented LMs largely outperform orders of magnitude larger LMs, while unassisted LMs remain competitive in questions about high-popularity entities.