Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring

This work develops a new transformer architecture, the Poly-encoder, that learns global rather than token level self-attention features and achieves state-of-the-art results on three existing tasks.