Language Understanding on MMLU (Accuracy and Refusal Rate)
74.7MMLU AccuracyOriginal
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OriginalModel Backbone=Qwen3-14B2026.05 | 74.7 | — | |
| ICCU (end-to-end)Model Backbone=Qwen3-14B2026.05 | 73.3 | 2.8 | |
| ICCU (filter + generate)Model Backbone=Qwen3-14B2026.05 | 73.1 | 3.3 | |
| O3Model Backbone=Qwen3-14B2026.05 | 61.9 | — | |
| OriginalModel Backbone=Llama-3-8B-Instruct2026.05 | 60.6 | — | |
| ICCU (filter + generate)Model Backbone=Llama-3-8B-Instruct2026.05 | 60.6 | 3.1 | |
| RMUModel Backbone=Qwen3-14B2026.05 | 59.9 | — | |
| ICCU (end-to-end)Model Backbone=Llama-3-8B-Instruct2026.05 | 58.4 | 5 | |
| O3Model Backbone=Llama-3-8B-Instruct2026.05 | 49.8 | — | |
| GAModel Backbone=Llama-3-8B-Instruct2026.05 | 30.2 | — | |
| RMUModel Backbone=Llama-3-8B-Instruct2026.05 | 26.4 | — | |
| GAModel Backbone=Qwen3-14B2026.05 | 23.9 | — |