Massive Multitask Language Understanding on MMLU (Performance Profile)
56.6MMLUQwen3-4B + FBS-Full (ours)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen3-4B + FBS-Full (ours)Backbone=Qwen3-4B2026.01 | 56.6 | 532 | 0.7 | 36 | |
| Qwen3-4B + FBS-S1Backbone=Qwen3-4B2026.01 | 56.4 | 755 | 1.03 | 0 | |
| Qwen3-4B-Instruct (Baseline)Backbone=Qwen3-4B2026.01 | 55.1 | 760 | 1 | 0 | |
| Qwen3-4B + EAGLE-2 (Group A)Backbone=Qwen3-4B2026.01 | 55 | 555 | 0.74 | 30 | |
| Qwen3-4B + Lookahead (Group A)Backbone=Qwen3-4B2026.01 | 55 | 595 | 0.82 | 15 | |
| Qwen3-4B + SpecDec (Group A)Backbone=Qwen3-4B2026.01 | 54.9 | 646 | 0.9 | 22 | |
| Qwen3-4B + Medusa (Group A)Backbone=Qwen3-4B2026.01 | 54.7 | 570 | 0.8 | 18 | |
| Base (no adapt.)Backbone=Llama-2-7B-Chat, Adaptation Protocol=No adaptation2026.04 | 46.1 | — | — | — | |
| SafeAnchorBackbone=Llama-2-7B-Chat, Adaptation Protocol=Sequential (Medical → Legal → Code)2026.04 | 45.7 | — | — | — | |
| EWC + LoRABackbone=Llama-2-7B-Chat, Adaptation Protocol=Sequential (Medical → Legal → Code)2026.04 | 45.3 | — | — | — | |
| Safety InterleavingBackbone=Llama-2-7B-Chat, Adaptation Protocol=Sequential (Medical → Legal → Code)2026.04 | 45.2 | — | — | — | |
| SafeGrad + LoRABackbone=Llama-2-7B-Chat, Adaptation Protocol=Sequential (Medical → Legal → Code)2026.04 | 45.1 | — | — | — | |
| O-LoRABackbone=Llama-2-7B-Chat, Adaptation Protocol=Sequential (Medical → Legal → Code)2026.04 | 45 | — | — | — | |
| Safe LoRABackbone=Llama-2-7B-Chat, Adaptation Protocol=Sequential (Medical → Legal → Code)2026.04 | 44.9 | — | — | — | |
| Standard LoRABackbone=Llama-2-7B-Chat, Adaptation Protocol=Sequential (Medical → Legal → Code)2026.04 | 44.8 | — | — | — | |
| Vaccine + LoRABackbone=Llama-2-7B-Chat, Adaptation Protocol=Sequential (Medical → Legal → Code)2026.04 | 44.5 | — | — | — |