Reading Comprehension on BoolQ (test)
99.87AccuracyMistral-Nemo 12B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Mistral-Nemo 12B2024.06 | 99.87 | 1.75 | |
| Gemma 7B2024.06 | 99.42 | 1.45 | |
| Llama-3 8B2024.06 | 99.3 | 1.37 | |
| Claude 3.5 Sonnet2024.06 | 98.81 | 1.2 | |
| Gemma-2 9B2024.06 | 98.28 | 1.3 | |
| Mistral 7B2024.06 | 98.07 | 1.41 | |
| GPT-4o2024.06 | 96.82 | 1.32 | |
| Queryable LoRAModel=LiquidAI/LFM2-700M2026.05 | 84.4 | — | |
| Queryable LoRAModel=Qwen/Qwen3-0.6B2026.05 | 84.4 | — | |
| LoRAModel=LiquidAI/LFM2-700M2026.05 | 81.2 | — | |
| Queryable LoRAModel=HuggingFaceTB/SmolLM2-360M-Instruct2026.05 | 78.1 | — | |
| Instruction-Queryable LoRAModel=HuggingFaceTB/SmolLM2-360M-Instruct2026.05 | 78.1 | — | |
| LoRAModel=Qwen/Qwen3-0.6B2026.05 | 78.1 | — | |
| Instruction-Queryable LoRAModel=Qwen/Qwen3-0.6B2026.05 | 78.1 | — | |
| Instruction-Queryable LoRAModel=LiquidAI/LFM2-700M2026.05 | 76.6 | — | |
| Instruction-Queryable LoRAModel=ibm-granite/granite-4.0-350m2026.05 | 76.6 | — | |
| GPT-3 175Balpha=-12021.10 | 73.7 | — | |
| Coherence Boosting (GPT-3 175B)alpha=-0.392021.10 | 72.69 | — | |
| LoRAModel=ibm-granite/granite-4.0-350m2026.05 | 71.9 | — | |
| GPT-3 175Balpha=02021.10 | 71.56 | — | |
| LoRAModel=Qwen/Qwen2.5-0.5B-Instruct2026.05 | 70.3 | — | |
| Queryable LoRAModel=Qwen/Qwen2.5-0.5B-Instruct2026.05 | 70.3 | — | |
| Instruction-Queryable LoRAModel=Qwen/Qwen2.5-0.5B-Instruct2026.05 | 70.3 | — | |
| Instruction-Queryable LoRAModel=amd/ReasonLite-0.6B-Turbo2026.05 | 70.3 | — | |
| Instruction-Queryable LoRAModel=LiquidAI/LFM2.5-350M2026.05 | 68.8 | — | |
| LoRAModel=Qwen/Qwen2.5-Coder-0.5B-Instruct2026.05 | 65.6 | — | |
| Instruction-Queryable LoRAModel=Qwen/Qwen2.5-Coder-0.5B-Instruct2026.05 | 65.6 | — | |
| Queryable LoRAModel=ibm-granite/granite-4.0-350m2026.05 | 65.6 | — | |
| LoRAModel=LiquidAI/LFM2.5-350M2026.05 | 64.1 | — | |
| GPT-2 XL (1.6B)alpha=-12021.10 | 63.46 | — | |
| Coherence Boosting (GPT-2 XL)alpha=-0.642021.10 | 63.21 | — | |
| LoRAModel=HuggingFaceTB/SmolLM2-360M-Instruct2026.05 | 62.5 | — | |
| Queryable LoRAModel=amd/ReasonLite-0.6B-Turbo2026.05 | 62.5 | — | |
| Coherence Boosting (GPT-2 Small)alpha=-3.042021.10 | 62.14 | — | |
| GPT-2 XL (1.6B)alpha=02021.10 | 62.14 | — | |
| Queryable LoRAModel=LiquidAI/LFM2.5-350M2026.05 | 59.4 | — | |
| Queryable LoRAModel=Qwen/Qwen2.5-Coder-0.5B-Instruct2026.05 | 59.4 | — | |
| LoRAModel=amd/ReasonLite-0.6B2026.05 | 59.4 | — | |
| Queryable LoRAModel=amd/ReasonLite-0.6B2026.05 | 59.4 | — | |
| GPT-2 Small (125M)alpha=-12021.10 | 58.07 | — | |
| LoRAModel=amd/ReasonLite-0.6B-Turbo2026.05 | 57.8 | — | |
| Instruction-Queryable LoRAModel=amd/ReasonLite-0.6B2026.05 | 56.2 | — | |
| GPT-2 Small (125M)alpha=02021.10 | 49.36 | — |