List your company
ResearchFormerly WizWand

Word Similarity on RG-65 (test)

0.848Spearman CorrelationFlan-T5-SD
0.580720.650110.71950.78889Sep 18, 2018Oct 22, 2019Nov 24, 2020Dec 28, 2021Jan 31, 2023Mar 5, 2024Apr 8, 2025
Updated 8mo ago

Evaluation results

MethodLinks
Flan-T5-SDBackbone=Flan-T5-xxl, Clusters (k)=52025.040.848
DRG2VecType=Discrete embeddings2025.040.845
SOTA (Single)Type=Special-purpose SOTA model (Single vector)2023.050.835
CBOWDim.=7682021.060.8348
SKIPGRAMDim.=7682021.060.8259
BERT-SDBackbone=BERT, Clusters (k)=52025.040.819
Dict2VecType=Discrete embeddings2025.040.814
ROBERTA2STATIC_paraParent Model=ROBERTA-12, Dim.=7682021.060.8057
ROBERTA2STATIC_sentParent Model=ROBERTA-12, Dim.=7682021.060.7999
GPT2STATIC_paraParent Model=GPT2-12, Dim.=7682021.060.7881
FRAGEFramework=with FRAGE2018.090.7878
SENT2VECDim.=7682021.060.7811
FASTTEXTDim.=300, training corpus relative size=12x2021.060.7669
GPT-J-6BCategory=Large existing models2023.050.766
BaselineFramework=Original2018.090.7563
BERT2STATIC_paraParent Model=BERT-12, Dim.=7682021.060.7555
Sim12Sense Selection=Sense 12 (relatedness)2023.050.754
GPT2STATIC_sentParent Model=GPT2-12, Dim.=7682021.060.7484
ASE - best layer per taskParent Model=BERT-12, Dim.=768, Layer=12021.060.7449
BERT2STATIC_sentParent Model=BERT-12, Dim.=7682021.060.7421
ASE - best layer per taskParent Model=GPT2-12, Dim.=768, Layer=12021.060.7013
ASE - best overall layerParent Model=BERT-12, Dim.=768, Layer=32021.060.6948
GloVeEmbedding Type=Classic Non-Contextual2023.050.687
ASE - best overall layerParent Model=GPT2-12, Dim.=768, Layer=22021.060.6833
word2vecEmbedding Type=Classic Non-Contextual2023.050.679
ASE - best layer per taskParent Model=ROBERTA-12, Dim.=768, Layer=02021.060.673
ASE - best overall layerParent Model=ROBERTA-12, Dim.=768, Layer=02021.060.673
GPT2-1.5BCategory=Large existing models2023.050.67
SimminSense Selection=Minimum Sense Cosine2023.050.653
GLOVEDim.=300, training corpus relative size=650x2021.060.6442
Trnsf 124MType=Baseline Transformer2023.050.634
Word2VecType=Discrete embeddings2025.040.617
Sim14Sense Selection=Sense 14 (Verb objects, nmod nouns)2023.050.591