Word Similarity on RG-65 (test)
0.848Spearman CorrelationFlan-T5-SD
Evaluation Results
| Method | Links | |
|---|---|---|
| Flan-T5-SDBackbone=Flan-T5-xxl, Clusters (k)=52025.04 | 0.848 | |
| DRG2VecType=Discrete embeddings2025.04 | 0.845 | |
| SOTA (Single)Type=Special-purpose SOTA model (Single vector)2023.05 | 0.835 | |
| CBOWDim.=7682021.06 | 0.8348 | |
| SKIPGRAMDim.=7682021.06 | 0.8259 | |
| BERT-SDBackbone=BERT, Clusters (k)=52025.04 | 0.819 | |
| Dict2VecType=Discrete embeddings2025.04 | 0.814 | |
| ROBERTA2STATIC_paraParent Model=ROBERTA-12, Dim.=7682021.06 | 0.8057 | |
| ROBERTA2STATIC_sentParent Model=ROBERTA-12, Dim.=7682021.06 | 0.7999 | |
| GPT2STATIC_paraParent Model=GPT2-12, Dim.=7682021.06 | 0.7881 | |
| FRAGEFramework=with FRAGE2018.09 | 0.7878 | |
| SENT2VECDim.=7682021.06 | 0.7811 | |
| FASTTEXTDim.=300, training corpus relative size=12x2021.06 | 0.7669 | |
| GPT-J-6BCategory=Large existing models2023.05 | 0.766 | |
| BaselineFramework=Original2018.09 | 0.7563 | |
| BERT2STATIC_paraParent Model=BERT-12, Dim.=7682021.06 | 0.7555 | |
| Sim12Sense Selection=Sense 12 (relatedness)2023.05 | 0.754 | |
| GPT2STATIC_sentParent Model=GPT2-12, Dim.=7682021.06 | 0.7484 | |
| ASE - best layer per taskParent Model=BERT-12, Dim.=768, Layer=12021.06 | 0.7449 | |
| BERT2STATIC_sentParent Model=BERT-12, Dim.=7682021.06 | 0.7421 | |
| ASE - best layer per taskParent Model=GPT2-12, Dim.=768, Layer=12021.06 | 0.7013 | |
| ASE - best overall layerParent Model=BERT-12, Dim.=768, Layer=32021.06 | 0.6948 | |
| GloVeEmbedding Type=Classic Non-Contextual2023.05 | 0.687 | |
| ASE - best overall layerParent Model=GPT2-12, Dim.=768, Layer=22021.06 | 0.6833 | |
| word2vecEmbedding Type=Classic Non-Contextual2023.05 | 0.679 | |
| ASE - best layer per taskParent Model=ROBERTA-12, Dim.=768, Layer=02021.06 | 0.673 | |
| ASE - best overall layerParent Model=ROBERTA-12, Dim.=768, Layer=02021.06 | 0.673 | |
| GPT2-1.5BCategory=Large existing models2023.05 | 0.67 | |
| SimminSense Selection=Minimum Sense Cosine2023.05 | 0.653 | |
| GLOVEDim.=300, training corpus relative size=650x2021.06 | 0.6442 | |
| Trnsf 124MType=Baseline Transformer2023.05 | 0.634 | |
| Word2VecType=Discrete embeddings2025.04 | 0.617 | |
| Sim14Sense Selection=Sense 14 (Verb objects, nmod nouns)2023.05 | 0.591 |