Word Similarity on WS-353 REL (test)
0.701Spearman CorrelationOurs (+post-process)
Evaluation Results
| Method | Links | |
|---|---|---|
| Ours (+post-process)Type=Context-to-Vec, Post-processing=With post-processing2022.10 | 0.701 | |
| Ours (preliminary)Type=Context-to-Vec, Post-processing=None2022.10 | 0.683 | |
| BERT-SDBackbone=BERT, Clusters (k)=52025.04 | 0.671 | |
| BERT+Skip-gramType=Context-to-Vec2022.10 | 0.667 | |
| FASTTEXTType=Static2022.10 | 0.616 | |
| DRG2VecType=Discrete embeddings2025.04 | 0.605 | |
| GPT2Type=Contextualized, Representation level=avg2022.10 | 0.597 | |
| Flan-T5-SDBackbone=Flan-T5-xxl, Clusters (k)=52025.04 | 0.584 | |
| XLNetType=Contextualized, Representation level=avg2022.10 | 0.58 | |
| Dict2VecType=Discrete embeddings2025.04 | 0.579 | |
| ContextLSTMType=Context-to-Vec2022.10 | 0.573 | |
| GPT2Type=Contextualized, Representation level=token2022.10 | 0.557 | |
| XLNetType=Contextualized, Representation level=word2022.10 | 0.555 | |
| Word2VecType=Discrete embeddings2025.04 | 0.548 | |
| CBOWType=Static2022.10 | 0.539 | |
| Skip-gramType=Static2022.10 | 0.537 | |
| XLNetType=Contextualized, Representation level=token2022.10 | 0.532 | |
| GloVeType=Static2022.10 | 0.502 | |
| BERTType=Contextualized, Representation level=avg2022.10 | 0.499 | |
| ELMoType=Contextualized, Representation level=avg2022.10 | 0.474 | |
| DepsType=Static2022.10 | 0.468 | |
| SynGCNType=Context-to-Vec2022.10 | 0.457 | |
| Skip-gram(context)Type=Static2022.10 | 0.435 | |
| BERTType=Contextualized, Representation level=token2022.10 | 0.425 | |
| ELMoType=Contextualized, Representation level=token2022.10 | 0.392 | |
| ELMoType=Contextualized, Representation level=word2022.10 | 0.324 | |
| GPT2Type=Contextualized, Representation level=word2022.10 | 0.276 | |
| BERTType=Contextualized, Representation level=word2022.10 | 0.141 |