Multi-subject Knowledge on MMLU
79.7AccuracyBaseline
Evaluation Results
| Method | Links | |
|---|---|---|
| BaselineModel=Qwen3-8B2026.05 | 79.7 | |
| ScaleSearchModel=Qwen3-8B2026.05 | 79.4 | |
| NVFP4Model=Qwen3-8B2026.05 | 77.7 | |
| TCA-AttentionBase Model=Qwen2.5-7B-Instruct2025.12 | 74.26 | |
| FlexPrefillBase Model=Qwen2.5-7B-Instruct2025.12 | 74.23 | |
| Qwen2.5-7B-InstructAttention Method=Full Attention2025.12 | 74.22 | |
| XAttentionBase Model=Qwen2.5-7B-Instruct2025.12 | 74.2 | |
| MInferenceBase Model=Qwen2.5-7B-Instruct2025.12 | 74.14 | |
| LLaMA3.1-8B-InstructAttention Method=Full Attention2025.12 | 69.38 | |
| XAttentionBase Model=LLaMA3.1-8B-Instruct2025.12 | 69.21 | |
| TCA-AttentionBase Model=LLaMA3.1-8B-Instruct2025.12 | 69.21 | |
| FlexPrefillBase Model=LLaMA3.1-8B-Instruct2025.12 | 69.16 | |
| MInferenceBase Model=LLaMA3.1-8B-Instruct2025.12 | 69.14 | |
| BaselineModel=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | 48 | |
| ScaleSearchModel=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | 45.4 | |
| NVFP4Model=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | 45.2 |