List your company
ResearchFormerly WizWand

Word Similarity on MEN

0.596Spearman RhoOmni TM-AE
0.0391840.1837420.32830.472858May 31, 2024Sep 25, 2024Jan 21, 2025May 19, 2025Sep 13, 2025Jan 9, 2026May 7, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Omni TM-AEHardware=CPU2026.050.596—0.439
FastOmniTMAEHardware=CPU2026.050.586—0.441
FastTextHardware=CPU2026.050.58—0.406
FastOmniTMAEHardware=CUDA2026.050.576—0.434
GloVeHardware=CPU2026.050.564—0.394
Word2VecHardware=CPU2026.050.561—0.391
ConceptsModel=Llama 3.1 8B, λ=1.02026.030.4362——
ConceptsModel=Llama 3.1 8B, λ=1.02026.030.4275——
ConceptsModel=Llama 3.1 8B, λ=0.752026.030.4193——
ConceptsModel=Llama 3.1 8B, λ=0.752026.030.4154——
ConceptsModel=Llama 3.1 8B, λ=0.52026.030.4086——
ConceptsModel=Llama 3.1 8B, λ=0.52026.030.4073——
ROOT-CAp=0, k (dimensionality)=500, Training Corpus (Text8, BNC, Wiki052024)=BNC2024.050.4——
ConceptsModel=Llama 3.1 8B, λ=0.252026.030.3987——
ConceptsModel=Llama 3.1 8B, λ=0.252026.030.3963——
ConceptsModel=Llama 3.2 3B, λ=1.02026.030.3936——
ConceptsModel=Llama 3.2 1B, λ=1.02026.030.3875——
BaselineModel=Llama 3.1 8B, λ=0.02026.030.383——
ConceptsModel=Llama 3.2 1B, λ=1.02026.030.3823——
ConceptsModel=Llama 3.1 8B, λ=0.02026.030.3818——
ConceptsModel=Llama 3.2 3B, λ=1.02026.030.3817——
ROOT-CAp=0.5, k (dimensionality)=800, Training Corpus (Text8, BNC, Wiki052024)=BNC2024.050.374——
ConceptsModel=Llama 3.2 1B, λ=0.752026.030.3679——
ConceptsModel=Llama 3.2 1B, λ=0.752026.030.3659——
ConceptsModel=Llama 3.2 3B, λ=0.752026.030.3649——
ConceptsModel=Llama 3.2 3B, λ=0.752026.030.3605——
BaselineModel=Llama 3.1 8B, λ=Base2026.030.3585——
BaselineModel=Llama 3.1 8B, λ=Base2026.030.3585——
ConceptsModel=Llama 3.2 1B, λ=0.52026.030.3527——
ConceptsModel=Llama 3.2 1B, λ=0.52026.030.3519——
ConceptsModel=Llama 3.2 3B, λ=0.52026.030.3507——
ConceptsModel=Llama 3.2 3B, λ=0.52026.030.3483——
NoiseModel=Llama 3.1 8B, λ=0.252026.030.3419——
ConceptsModel=Llama 3.2 3B, λ=0.252026.030.3405——
ROOTROOT-CAp=0, k (dimensionality)=600, Training Corpus (Text8, BNC, Wiki052024)=Text82024.050.34——
ConceptsModel=Llama 3.2 3B, λ=0.252026.030.3372——
ConceptsModel=Llama 3.2 1B, λ=0.252026.030.3366——
ConceptsModel=Llama 3.2 1B, λ=0.252026.030.3365——
NoiseModel=Llama 3.1 8B, λ=0.252026.030.3339——
ConceptsModel=Llama 3.2 3B, λ=0.02026.030.327——
ROOT-CAp=0, k (dimensionality)=800, Training Corpus (Text8, BNC, Wiki052024)=Text82024.050.325——
ROOT-CAp=0.5, k (dimensionality)=9000, Training Corpus (Text8, BNC, Wiki052024)=Text82024.050.324——
NoiseModel=Llama 3.1 8B, λ=0.52026.030.3225——
BaselineModel=Llama 3.2 3B, λ=0.02026.030.3214——
BaselineModel=Llama 3.2 1B, λ=Base2026.030.3196——
BaselineModel=Llama 3.2 1B, λ=Base2026.030.3196——
BaselineModel=Llama 3.2 1B, λ=0.02026.030.3182——
BaselineModel=Llama 3.2 3B, λ=Base2026.030.3173——
BaselineModel=Llama 3.2 3B, λ=Base2026.030.3173——
ConceptsModel=Llama 3.2 1B, λ=0.02026.030.3162——
NoiseModel=Llama 3.1 8B, λ=0.52026.030.3099——
NoiseModel=Llama 3.2 1B, λ=0.252026.030.2969——
NoiseModel=Llama 3.2 1B, λ=0.252026.030.2941——
RAW-CAp=0, k (dimensionality)=600, Training Corpus (Text8, BNC, Wiki052024)=BNC2024.050.293——
NoiseModel=Llama 3.2 3B, λ=0.252026.030.291——
NoiseModel=Llama 3.2 3B, λ=0.252026.030.2908——
NoiseModel=Llama 3.2 1B, λ=0.52026.030.2866——
NoiseModel=Llama 3.1 8B, λ=0.752026.030.2846——
NoiseModel=Llama 3.1 8B, λ=0.752026.030.2796——
NoiseModel=Llama 3.2 3B, λ=0.52026.030.2771——
NoiseModel=Llama 3.2 1B, λ=0.52026.030.2736——
NoiseModel=Llama 3.2 3B, λ=0.52026.030.2663——
NoiseModel=Llama 3.2 1B, λ=0.752026.030.2589——
NoiseModel=Llama 3.2 3B, λ=0.752026.030.2489——
NoiseModel=Llama 3.2 1B, λ=0.752026.030.2447——
NoiseModel=Llama 3.2 3B, λ=0.752026.030.2366——
RAW-CAp=0, k (dimensionality)=300, Training Corpus (Text8, BNC, Wiki052024)=Text82024.050.223——
RAW-CAp=0, k (dimensionality)=1000, Training Corpus (Text8, BNC, Wiki052024)=Wiki0520242024.050.19——
NoiseModel=Llama 3.1 8B, λ=1.02026.030.0909——
NoiseModel=Llama 3.1 8B, λ=1.02026.030.0811——
NoiseModel=Llama 3.2 1B, λ=1.02026.030.0753——
NoiseModel=Llama 3.2 1B, λ=1.02026.030.0748——
NoiseModel=Llama 3.2 3B, λ=1.02026.030.0634——
NoiseModel=Llama 3.2 3B, λ=1.02026.030.0606——
RAW-CAp=0, Training Corpus (Text8, BNC, Wiki052024)=Total2024.05—0.706—