Task-Oriented Intrinsic Evaluation of Semantic Textual Similarity

This work defines how the validity of an intrinsic evaluation can be assessed and compares different intrinsic evaluation methods and proposes a framework for conducting the intrinsic evaluation which takes the properties of the targeted task into account.