Downstream Performance Evaluation on CORE
19.94CORE ScoreEC
Evaluation Results
| Method | Links | |
|---|---|---|
| ECBatch=512k2026.03 | 19.94 | |
| ETBatch=0.5M → 500M, Hyperparameter=β=0.999+warmup2026.03 | 19.876 | |
| ECBatch=8k2026.03 | 18.83 | |
| ECBatch=64k2026.03 | 18.754 | |
| TC loss-freeBatch=512k2026.03 | 18.031 | |
| TCBatch=—2026.03 | 17.983 | |
| ECBatch=2k2026.03 | 17.91 | |
| KromHCD (Number of transformer blocks)=12, n (Number of residual streams)=4, ∆ Params (K)=9592026.01 | 16.872 | |
| mHCD (Number of transformer blocks)=12, n (Number of residual streams)=4, ∆ Params (K)=18442026.01 | 16.023 | |
| TC auxBatch=64k2026.03 | 15.894 | |
| denseBatch=—2026.03 | 15.743 | |
| ResidualD (Number of transformer blocks)=12, n (Number of residual streams)=42026.01 | 14.774 | |
| mHC-liteD (Number of transformer blocks)=12, n (Number of residual streams)=4, ∆ Params (K)=24332026.01 | 13.217 | |
| KromHCD (Number of transformer blocks)=6, n (Number of residual streams)=4, ∆ Params (K)=2402026.01 | 9.018 | |
| mHC-liteD (Number of transformer blocks)=6, n (Number of residual streams)=4, ∆ Params (K)=6092026.01 | 8.208 | |
| mHCD (Number of transformer blocks)=6, n (Number of residual streams)=4, ∆ Params (K)=4622026.01 | 7.971 | |
| ResidualD (Number of transformer blocks)=6, n (Number of residual streams)=42026.01 | 6.477 | |
| DetVocabulary Size=128k, Depth=242026.05 | 0.302 | |
| BPEVocabulary Size=128k, Depth=242026.05 | 0.296 | |
| BiasVocabulary Size=128k, Depth=242026.05 | 0.296 | |
| BiasVocabulary Size=32k, Depth=242026.05 | 0.287 | |
| BPEVocabulary Size=32k, Depth=242026.05 | 0.279 | |
| DetVocabulary Size=32k, Depth=242026.05 | 0.278 | |
| BPEVocabulary Size=8k, Depth=242026.05 | 0.27 | |
| BiasVocabulary Size=8k, Depth=242026.05 | 0.265 | |
| DetVocabulary Size=8k, Depth=242026.05 | 0.251 | |
| BiasVocabulary Size=128k, Depth=182026.05 | 0.238 | |
| BPEVocabulary Size=128k, Depth=182026.05 | 0.233 | |
| BPEVocabulary Size=32k, Depth=182026.05 | 0.232 | |
| DetVocabulary Size=128k, Depth=182026.05 | 0.229 | |
| BiasVocabulary Size=32k, Depth=182026.05 | 0.22 | |
| DetVocabulary Size=32k, Depth=182026.05 | 0.22 | |
| BPEVocabulary Size=8k, Depth=182026.05 | 0.195 | |
| BiasVocabulary Size=8k, Depth=182026.05 | 0.195 | |
| DetVocabulary Size=8k, Depth=182026.05 | 0.186 | |
| DetVocabulary Size=128k, Depth=122026.05 | 0.17 | |
| DetVocabulary Size=64k, Depth=122026.05 | 0.165 | |
| BiasVocabulary Size=64k, Depth=122026.05 | 0.164 | |
| BiasVocabulary Size=256k, Depth=122026.05 | 0.163 | |
| BPEVocabulary Size=128k, Depth=122026.05 | 0.161 | |
| BPEVocabulary Size=256k, Depth=122026.05 | 0.159 | |
| BPEVocabulary Size=64k, Depth=122026.05 | 0.158 | |
| DetVocabulary Size=256k, Depth=122026.05 | 0.158 | |
| BiasVocabulary Size=128k, Depth=122026.05 | 0.156 | |
| DetVocabulary Size=32k, Depth=122026.05 | 0.151 | |
| BPEVocabulary Size=32k, Depth=122026.05 | 0.15 | |
| BiasVocabulary Size=32k, Depth=122026.05 | 0.148 | |
| BiasVocabulary Size=16k, Depth=122026.05 | 0.144 | |
| BPEVocabulary Size=16k, Depth=122026.05 | 0.139 | |
| DetVocabulary Size=16k, Depth=122026.05 | 0.137 | |
| BPEVocabulary Size=8k, Depth=122026.05 | 0.1345 | |
| BiasVocabulary Size=8k, Depth=122026.05 | 0.1318 | |
| DetVocabulary Size=8k, Depth=122026.05 | 0.1294 |