Word Similarity on SimLex999 (test)
0.789Spearman CorrelationAUXGAN
Evaluation Results
| Method | Links | |
|---|---|---|
| AUXGANVector space=GLOVE-CC2018.09 | 0.789 | |
| POST-DFFNVector space=GLOVE-CC2018.09 | 0.785 | |
| AUXGANVector space=SGNS-W22018.09 | 0.782 | |
| ATTRACT-REPELVector space=GLOVE-CC2018.09 | 0.781 | |
| POST-DFFNVector space=SGNS-W22018.09 | 0.781 | |
| ATTRACT-REPELVector space=SGNS-W22018.09 | 0.778 | |
| POST-DFFNVector space=FASTTEXT2018.09 | 0.768 | |
| AUXGANVector space=FASTTEXT2018.09 | 0.766 | |
| ATTRACT-REPELVector space=FASTTEXT2018.09 | 0.761 | |
| SOTA (Multi)Type=Special-purpose SOTA model (Multi vector)2023.05 | 0.605 | |
| Flan-T5-SDBackbone=Flan-T5-xxl, Clusters (k)=52025.04 | 0.555 | |
| SOTA (Single)Type=Special-purpose SOTA model (Single vector)2023.05 | 0.554 | |
| ROBERTA2STATIC_sentParent Model=ROBERTA-12, Dim.=7682021.06 | 0.5507 | |
| BERT2STATIC_sentParent Model=BERT-12, Dim.=7682021.06 | 0.5461 | |
| GPT2STATIC_paraParent Model=GPT2-12, Dim.=7682021.06 | 0.5417 | |
| SimminSense Selection=Minimum Sense Cosine2023.05 | 0.54 | |
| GPT2-1.5BCategory=Large existing models2023.05 | 0.523 | |
| Sim12Sense Selection=Sense 12 (relatedness)2023.05 | 0.522 | |
| Sim14Sense Selection=Sense 14 (Verb objects, nmod nouns)2023.05 | 0.5 | |
| BERT-SDBackbone=BERT, Clusters (k)=52025.04 | 0.499 | |
| GPT-J-6BCategory=Large existing models2023.05 | 0.492 | |
| Trnsf 124MType=Baseline Transformer2023.05 | 0.478 | |
| DRG2VecType=Discrete embeddings2025.04 | 0.476 | |
| Dict2VecType=Discrete embeddings2025.04 | 0.444 | |
| word2vecEmbedding Type=Classic Non-Contextual2023.05 | 0.442 | |
| Distributional (X)Vector space=SGNS-W22018.09 | 0.414 | |
| Distributional (X)Vector space=GLOVE-CC2018.09 | 0.407 | |
| Distributional (X)Vector space=FASTTEXT2018.09 | 0.383 | |
| GloVeEmbedding Type=Classic Non-Contextual2023.05 | 0.371 | |
| Word2VecType=Discrete embeddings2025.04 | 0.338 |