Stranger than Paradigms Word Embedding Benchmarks Don’t Align With Morphology

This study empirically test the hypothesis that models that perform well on morphology tasks tend to differ from those which succeed on more traditional benchmarks by exhaustively evaluating 1,200 French models with varying hyperparameters on 14 different tasks.