Word Similarity on SimVerb-3500 (test)
0.764Spearman CorrelationPOST-DFFN
Evaluation Results
| Method | Links | |
|---|---|---|
| POST-DFFNVector space=GLOVE-CC2018.09 | 0.764 | |
| AUXGANVector space=GLOVE-CC2018.09 | 0.764 | |
| POST-DFFNVector space=SGNS-W22018.09 | 0.763 | |
| AUXGANVector space=SGNS-W22018.09 | 0.762 | |
| ATTRACT-REPELVector space=GLOVE-CC2018.09 | 0.761 | |
| ATTRACT-REPELVector space=SGNS-W22018.09 | 0.761 | |
| POST-DFFNVector space=FASTTEXT2018.09 | 0.745 | |
| ATTRACT-REPELVector space=FASTTEXT2018.09 | 0.744 | |
| AUXGANVector space=FASTTEXT2018.09 | 0.741 | |
| SOTA (Multi)Type=Special-purpose SOTA model (Multi vector)2023.05 | 0.528 | |
| Sim14Sense Selection=Sense 14 (Verb objects, nmod nouns)2023.05 | 0.502 | |
| SOTA (Single)Type=Special-purpose SOTA model (Single vector)2023.05 | 0.473 | |
| Sim12Sense Selection=Sense 12 (relatedness)2023.05 | 0.471 | |
| SimminSense Selection=Minimum Sense Cosine2023.05 | 0.471 | |
| Flan-T5-SDBackbone=Flan-T5-xxl, Clusters (k)=52025.04 | 0.439 | |
| DRG2VecType=Discrete embeddings2025.04 | 0.425 | |
| BERT-SDBackbone=BERT, Clusters (k)=52025.04 | 0.425 | |
| GPT2-1.5BCategory=Large existing models2023.05 | 0.418 | |
| Dict2VecType=Discrete embeddings2025.04 | 0.379 | |
| GPT-J-6BCategory=Large existing models2023.05 | 0.374 | |
| word2vecEmbedding Type=Classic Non-Contextual2023.05 | 0.367 | |
| Trnsf 124MType=Baseline Transformer2023.05 | 0.363 | |
| Distributional (X)Vector space=GLOVE-CC2018.09 | 0.28 | |
| Distributional (X)Vector space=SGNS-W22018.09 | 0.272 | |
| Distributional (X)Vector space=FASTTEXT2018.09 | 0.247 | |
| GloVeEmbedding Type=Classic Non-Contextual2023.05 | 0.227 | |
| Word2VecType=Discrete embeddings2025.04 | 0.19 |