Question Answering on MathQA, MMLU, HellaSwag, RACE, and CosmosQA
0.079ECER1 Llama
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| R1 LlamaProtocol=N-gram Suppression, Model Category=Reasoning Model2026.04 | 0.079 | 0.079 | 0.769 | 25.1 | 52.8 | |
| Llama 3.3Protocol=Baseline Inference, Model Category=Instruction-tuned Model2026.04 | 0.121 | 0.121 | 0.685 | 38.2 | 73.3 | |
| Phi-4 miniProtocol=N-gram Suppression, Model Category=Instruction-tuned Model2026.04 | 0.128 | 0.127 | 0.72 | 65.4 | 71.1 | |
| Phi-4 miniProtocol=N-gram Injection, Model Category=Instruction-tuned Model2026.04 | 0.132 | 0.132 | 0.809 | 24.5 | 51 | |
| Llama 3.3Protocol=N-gram Suppression, Model Category=Instruction-tuned Model2026.04 | 0.146 | 0.146 | 0.774 | 28.2 | 59.9 | |
| Llama 3.3Protocol=N-gram Injection, Model Category=Instruction-tuned Model2026.04 | 0.148 | 0.148 | 0.802 | 24 | 52.2 | |
| R1 Qwen3Protocol=N-gram Suppression, Model Category=Reasoning Model2026.04 | 0.166 | 0.166 | 0.344 | 82.6 | 56.6 | |
| R1 Qwen3Protocol=N-gram Injection, Model Category=Reasoning Model2026.04 | 0.167 | 0.166 | 0.359 | 81.5 | 56.6 | |
| GPT-ossProtocol=Baseline Inference, Model Category=Reasoning Model2026.04 | 0.191 | 0.191 | 0.412 | 78.3 | 53.2 | |
| GPT-ossProtocol=N-gram Suppression, Model Category=Reasoning Model2026.04 | 0.191 | 0.191 | 0.401 | 79.5 | 54.6 | |
| Phi-4Protocol=N-gram Suppression, Model Category=Reasoning Model2026.04 | 0.194 | 0.194 | 0.448 | 73.4 | 68.9 | |
| GPT-ossProtocol=N-gram Injection, Model Category=Reasoning Model2026.04 | 0.197 | 0.197 | 0.412 | 79.1 | 52.6 | |
| R1 Qwen3Protocol=Baseline Inference, Model Category=Reasoning Model2026.04 | 0.198 | 0.198 | 0.45 | 75.9 | 60.4 | |
| Phi-4Protocol=Baseline Inference, Model Category=Reasoning Model2026.04 | 0.199 | 0.199 | 0.405 | 79.5 | 56.2 | |
| Qwen 2.5Protocol=N-gram Suppression, Model Category=Instruction-tuned Model2026.04 | 0.205 | 0.205 | 0.657 | 50.1 | 77.7 | |
| Phi-4Protocol=N-gram Injection, Model Category=Reasoning Model2026.04 | 0.209 | 0.209 | 0.486 | 71.4 | 67.2 | |
| R1 LlamaProtocol=Baseline Inference, Model Category=Reasoning Model2026.04 | 0.245 | 0.245 | 0.615 | 62 | 69.1 | |
| Qwen 2.5Protocol=N-gram Injection, Model Category=Instruction-tuned Model2026.04 | 0.245 | 0.245 | 0.849 | 29.7 | 60 | |
| R1 LlamaProtocol=N-gram Injection, Model Category=Reasoning Model2026.04 | 0.267 | 0.267 | 0.723 | 48.8 | 72.8 | |
| Phi-4 miniProtocol=Baseline Inference, Model Category=Instruction-tuned Model2026.04 | 0.285 | 0.283 | 0.64 | 66 | 50.9 | |
| R1 QwenProtocol=N-gram Injection, Model Category=Reasoning Model2026.04 | 0.312 | 0.31 | 0.704 | 61.6 | 54.2 | |
| R1 QwenProtocol=N-gram Suppression, Model Category=Reasoning Model2026.04 | 0.318 | 0.316 | 0.7 | 63 | 51 | |
| Qwen 2.5Protocol=Baseline Inference, Model Category=Instruction-tuned Model2026.04 | 0.329 | 0.329 | 0.944 | 27.4 | 58.7 | |
| R1 QwenProtocol=Baseline Inference, Model Category=Reasoning Model2026.04 | 0.334 | 0.334 | 0.713 | 62.7 | 50.2 | |
| NemotronProtocol=N-gram Suppression, Model Category=Reasoning Model2026.04 | 0.334 | 0.334 | 0.859 | 55.3 | 63.7 | |
| NemotronProtocol=N-gram Injection, Model Category=Reasoning Model2026.04 | 0.409 | 0.409 | 0.898 | 49.6 | 54.4 | |
| NemotronProtocol=Baseline Inference, Model Category=Reasoning Model2026.04 | 0.415 | 0.415 | 0.896 | 51 | 51.7 |