List your company
ResearchFormerly WizWand

Downstream Performance Evaluation on CORE

19.94CORE ScoreEC
-0.6630244.68583810.034715.383562Jan 29, 2026Feb 16, 2026Mar 7, 2026Mar 26, 2026Apr 13, 2026May 2, 2026May 21, 2026
Updated 5mo ago

Evaluation results

MethodLinks
ECBatch=512k2026.0319.94
ETBatch=0.5M → 500M, Hyperparameter=β=0.999+warmup2026.0319.876
ECBatch=8k2026.0318.83
ECBatch=64k2026.0318.754
TC loss-freeBatch=512k2026.0318.031
TCBatch=—2026.0317.983
ECBatch=2k2026.0317.91
KromHCD (Number of transformer blocks)=12, n (Number of residual streams)=4, ∆ Params (K)=9592026.0116.872
mHCD (Number of transformer blocks)=12, n (Number of residual streams)=4, ∆ Params (K)=18442026.0116.023
TC auxBatch=64k2026.0315.894
denseBatch=—2026.0315.743
ResidualD (Number of transformer blocks)=12, n (Number of residual streams)=42026.0114.774
mHC-liteD (Number of transformer blocks)=12, n (Number of residual streams)=4, ∆ Params (K)=24332026.0113.217
KromHCD (Number of transformer blocks)=6, n (Number of residual streams)=4, ∆ Params (K)=2402026.019.018
mHC-liteD (Number of transformer blocks)=6, n (Number of residual streams)=4, ∆ Params (K)=6092026.018.208
mHCD (Number of transformer blocks)=6, n (Number of residual streams)=4, ∆ Params (K)=4622026.017.971
ResidualD (Number of transformer blocks)=6, n (Number of residual streams)=42026.016.477
DetVocabulary Size=128k, Depth=242026.050.302
BPEVocabulary Size=128k, Depth=242026.050.296
BiasVocabulary Size=128k, Depth=242026.050.296
BiasVocabulary Size=32k, Depth=242026.050.287
BPEVocabulary Size=32k, Depth=242026.050.279
DetVocabulary Size=32k, Depth=242026.050.278
BPEVocabulary Size=8k, Depth=242026.050.27
BiasVocabulary Size=8k, Depth=242026.050.265
DetVocabulary Size=8k, Depth=242026.050.251
BiasVocabulary Size=128k, Depth=182026.050.238
BPEVocabulary Size=128k, Depth=182026.050.233
BPEVocabulary Size=32k, Depth=182026.050.232
DetVocabulary Size=128k, Depth=182026.050.229
BiasVocabulary Size=32k, Depth=182026.050.22
DetVocabulary Size=32k, Depth=182026.050.22
BPEVocabulary Size=8k, Depth=182026.050.195
BiasVocabulary Size=8k, Depth=182026.050.195
DetVocabulary Size=8k, Depth=182026.050.186
DetVocabulary Size=128k, Depth=122026.050.17
DetVocabulary Size=64k, Depth=122026.050.165
BiasVocabulary Size=64k, Depth=122026.050.164
BiasVocabulary Size=256k, Depth=122026.050.163
BPEVocabulary Size=128k, Depth=122026.050.161
BPEVocabulary Size=256k, Depth=122026.050.159
BPEVocabulary Size=64k, Depth=122026.050.158
DetVocabulary Size=256k, Depth=122026.050.158
BiasVocabulary Size=128k, Depth=122026.050.156
DetVocabulary Size=32k, Depth=122026.050.151
BPEVocabulary Size=32k, Depth=122026.050.15
BiasVocabulary Size=32k, Depth=122026.050.148
BiasVocabulary Size=16k, Depth=122026.050.144
BPEVocabulary Size=16k, Depth=122026.050.139
DetVocabulary Size=16k, Depth=122026.050.137
BPEVocabulary Size=8k, Depth=122026.050.1345
BiasVocabulary Size=8k, Depth=122026.050.1318
DetVocabulary Size=8k, Depth=122026.050.1294