Entropy– and Distance-Regularized Attention Improves Low-Resource Neural Machine Translation
Entropy– and Distance-Regularized Attention Improves Low-Resource Neural Machine Translation
Ali Araabi,Vlad Niculae,C. Monz
2024 · DBLP: conf/amta/AraabiNM24
Conference of the Association for Machine Translation in the Americas · 4 Citations
TLDR
EaDRA (Entropy– and Distance-Regularized Attention), which introduces an inductive bias to prioritize essential elements and guide the attention mechanism accordingly, demonstrates significant improvements in translation quality, while incurring negligible computational cost.