List your company
ResearchFormerly WizWand

Word Similarity on RG-65

0.851Spearman CorrelationOurs (+post-process)
0.309160.449830.59050.73117Oct 17, 2018Jun 19, 2019Feb 20, 2020Oct 23, 2020Jun 25, 2021Feb 26, 2022Oct 30, 2022
Updated 6mo ago

Evaluation results

MethodLinks
Ours (+post-process)Type=Context-to-Vec2022.100.851
Ours (preliminary)Type=Context-to-Vec2022.100.843
CBOWDim.=768, Source=trained by us2021.060.8348
SKIPGRAMDim.=768, Source=trained by us2021.060.8259
BERT (avg)Type=Contextualized, Representation level=avg2022.100.812
BERT2STATICparaParent Model=BERT-24, Dim.=10242021.060.8085
FASTTEXTType=Static2022.100.808
ROBERTA2STATICparaParent Model=ROBERTA-12, Dim.=7682021.060.8057
BERT2STATICsentParent Model=BERT-24, Dim.=10242021.060.8031
ROBERTA2STATICsentParent Model=ROBERTA-12, Dim.=7682021.060.7999
ROBERTA2STATICparaParent Model=ROBERTA-24, Dim.=10242021.060.7939
GPT22STATICparaParent Model=GPT2-24, Dim.=10242021.060.7907
GPT22STATICparaParent Model=GPT2-12, Dim.=7682021.060.7881
BERT+Skip-gramType=Context-to-Vec2022.100.786
GPT22STATICsentParent Model=GPT2-24, Dim.=10242021.060.7815
SENT2VECDim.=768, Source=trained by us2021.060.7811
ASE - best layer per taskParent Model=BERT-24, Dim.=1024, Layer=92021.060.7745
ASE - best task independent layerParent Model=BERT-24, Dim.=1024, Layer=72021.060.7677
ROBERTA2STATICsentParent Model=ROBERTA-24, Dim.=10242021.060.7677
FASTTEXTDim.=300, Size of training corpus relative to ours=12x2021.060.7669
BERT2STATICparaParent Model=BERT-12, Dim.=7682021.060.7555
Skip-gramType=Static2022.100.752
GPT22STATICsentParent Model=GPT2-12, Dim.=7682021.060.7484
ASE - best layer per taskParent Model=BERT-12, Dim.=768, Layer=12021.060.7449
BERT2STATICsentParent Model=BERT-12, Dim.=7682021.060.7421
ASE - best layer per taskParent Model=GPT2-12, Dim.=768, Layer=12021.060.7013
ASE - best overall layerParent Model=BERT-12, Dim.=768, Layer=32021.060.6948
W2GEmbedding Method=W2G, Training Corpus=ukWaC and WaCkypedia2018.100.69
ASE - best overall layerParent Model=GPT2-12, Dim.=768, Layer=22021.060.6833
ASE - best layer per taskParent Model=ROBERTA-24, Dim.=1024, Layer=82021.060.6782
ASE - best task independent layerParent Model=ROBERTA-24, Dim.=1024, Layer=62021.060.6738
ASE - best layer per taskParent Model=ROBERTA-12, Dim.=768, Layer=02021.060.673
ASE - best overall layerParent Model=ROBERTA-12, Dim.=768, Layer=02021.060.673
ASE - best layer per taskParent Model=GPT2-24, Dim.=1024, Layer=12021.060.6574
EllEmbedding Method=Ell, Training Corpus=ukWaC and WaCkypedia2018.100.65
GLOVEDim.=300, Size of training corpus relative to ours=650x2021.060.6442
GloVe+GTEmbedding Method=GloVe, Transform=Gaussian Transform, Refinement Corpus=text82018.100.62
GloVeEmbedding Method=GloVe2018.100.6
ASE - best task independent layerParent Model=GPT2-24, Dim.=1024, Layer=132021.060.5773
W2V*text8Embedding Method=W2V, Training Corpus=text82018.100.56
GloVe*text8Embedding Method=GloVe, Training Corpus=text82018.100.33