Jointly optimizing word representations for lexical and sentential tasks with the C-PHRASE model
Jointly optimizing word representations for lexical and sentential tasks with the C-PHRASE model
N. Pham,Germán Kruszewski,Angeliki Lazaridou,Marco Baroni
TLDR
C-PHRASE, a distributional semantic model that learns word representations by optimizing context prediction for phrases at all levels in a syntactic tree, outperforms the state-of-theart C-BOW model on a variety of lexical tasks.
Abstract
We introduce C-PHRASE, a distributional semantic model that learns word representations by optimizing context prediction for phrases at all levels in a syntactic tree, from single words to full sentences. C-PHRASE outperforms the state-of-theart C-BOW model on a variety of lexical tasks. Moreover, since C-PHRASE word vectors are induced through a compositional learning objective (modeling the contexts of words combined into phrases), when they are summed, they produce sentence representations that rival those generated by ad-hoc compositional models.