Comparing Contextual and Static Word Embeddings with Small Data

Results show that contextualized models provide better word embeddings than static models and that merging embeddings from different models boosts task performance.