Natural Language Understanding on GLUE (QQP, QNLI, SST-2)
39.93QQP AccuracyRFMoE
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| RFMoEScale=L, Size=870.6M, FLOPs=613.2M, Zero-shot=true2026.04 | 39.93 | 49.86 | 53.67 | |
| MoEScale=L, Size=808.4M, FLOPs=608.4M, Zero-shot=true2026.04 | 37.57 | 49.28 | 49.08 | |
| RFMoEScale=M, Size=307.3M, FLOPs=249.2M, Zero-shot=true2026.04 | 37.06 | 49.51 | 57.34 | |
| RFMoEScale=S, Size=95.32M, FLOPs=91.08M, Zero-shot=true2026.04 | 36.98 | 49.44 | 53.56 | |
| MoEScale=M, Size=289.9M, FLOPs=248.0M, Zero-shot=true2026.04 | 36.85 | 49.46 | 49.31 | |
| ReMoEScale=S, Size=92.44M, FLOPs=90.93M, Zero-shot=true2026.04 | 36.83 | 49.48 | 49.08 | |
| MoEScale=S, Size=92.44M, FLOPs=90.93M, Zero-shot=true2026.04 | 36.82 | 49.46 | 49.08 | |
| AoEScale=S, Size=93.85M, FLOPs=88.57M, Zero-shot=true2026.04 | 36.82 | 49.46 | 49.08 |