Why does PairDiff work? - A Mathematical Analysis of Bilinear Relational Compositional Operators for Analogy Detection

A theoretical analysis of generalised bilinear operators that can be used to measure the l2 relational distance between two word-pairs shows that, if the word embed- dings are standardised and uncorrelated, such an operator will be independent of bil inear terms, and can be simplified to a linear form, where PairDiff is a special case.