Text Classification on GLUE (CoLA, SST-2, MRPC, STSB, RTE, QNLI, MNLI)
95.06SST-2 AccuracyNAMEx-Full-Mom
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| NAMEx-Full-MomParams=1.0B, Momentum=true2025.10 | 95.06 | 93.27 | 60.13 | 90.63 | 78.15 | 93.31 | 87.45 | |
| NAMEx-FullParams=1.0B, Momentum=false2025.10 | 94.82 | 92.8 | 59.63 | 90.27 | 77.83 | 93.23 | 87.23 | |
| EP-CAMEx-MomParams=1.0B, Momentum=true2025.10 | 94.61 | 92.47 | 59.31 | 90.07 | 76.17 | 92.99 | 86.8 | |
| NAMEx-MomParams=1.0B, Momentum=true2025.10 | 94.61 | 93.02 | 58.9 | 90.06 | 77.62 | 93.11 | 87.02 | |
| NAMExParams=1.0B, Momentum=false2025.10 | 94.46 | 92.01 | 58.81 | 90.12 | 75.09 | 92.86 | 86.96 | |
| SMoE (Top-2)Params=1.0B2025.10 | 94.35 | 91.04 | 58.43 | 89.73 | 74.98 | 92.48 | 86.72 | |
| SMoE (Top-1)Params=1.0B2025.10 | 94.26 | 90.87 | 56.78 | 89.44 | 70.75 | 92.07 | 86.38 | |
| CAMExParams=1.0B2025.10 | 93.8 | 91.16 | 58.57 | 89.47 | 74.72 | 92.6 | 86.44 | |
| EP-CAMExParams=1.0B, Momentum=false2025.10 | 93.69 | 91.01 | 58.29 | 89.92 | 75.81 | 92.17 | 86.94 | |
| DenseParams=220M2025.10 | 93.34 | 89.7 | 58.06 | 89.06 | 74.36 | 92.34 | 86.36 |