Chinese Multiple-Choice Evaluation on CMMLU
61.9CMMLU College Mathematics AccuracyBaseline (FlashAttention-2)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Baseline (FlashAttention-2)Model=Qwen3-30B, Evaluation protocol=fixed few-shot2026.04 | 61.9 | 85.71 | |
| VFAModel=Qwen3-30B, Evaluation protocol=fixed few-shot2026.04 | 61.9 | 84.76 | |
| VFAModel=Qwen3-8B, Evaluation protocol=fixed few-shot2026.04 | 60 | 81.9 | |
| Baseline (FlashAttention-2)Model=Qwen3-8B, Evaluation protocol=fixed few-shot2026.04 | 59.05 | 82.86 | |
| VFAModel=Llama3-8B, Evaluation protocol=fixed few-shot2026.04 | 39.05 | 61.9 | |
| Baseline (FlashAttention-2)Model=Llama3-8B, Evaluation protocol=fixed few-shot2026.04 | 38.1 | 62.86 |