Boolean Question Answering on BoolQ (test)
86.7Accuracy (Avg)QWEN3-14B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| QWEN3-14BModel Architecture=QWEN3-14B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 86.7 | — | — | — | — | — | |
| MISTRAL-7BModel Architecture=MISTRAL-7B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 86.42 | — | — | — | — | — | |
| Best ModelBackbone LLM=LLaMA 3.1 8B, Shot count=0-shot2025.10 | 86.2 | — | — | — | 3 | -8.8 | |
| DEEPSEEK-R1-QWEN-8BModel Architecture=DEEPSEEK-R1-QWEN-8B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 84.92 | — | — | — | — | — | |
| DEEPSEEK-R1-LLAMA-8BModel Architecture=DEEPSEEK-R1-LLAMA-8B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 83.43 | — | — | — | — | — | |
| QWEN3-4BModel Architecture=QWEN3-4B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 83.33 | — | — | — | — | — | |
| LLAMA-3.1-8BModel Architecture=LLAMA-3.1-8B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 83.09 | — | — | — | — | — | |
| DARTModel Architecture=MISTRAL-7B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 78.87 | — | — | — | — | — | |
| Best ModelBackbone LLM=Lucie 7B, Shot count=0-shot2025.10 | 77.5 | — | — | — | 5 | -17.2 | |
| DARTModel Architecture=DEEPSEEK-R1-LLAMA-8B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 75.47 | — | — | — | — | — | |
| LLAMA-3.2-3BModel Architecture=LLAMA-3.2-3B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 74.04 | — | — | — | — | — | |
| CONDACCBackbone=OPT-13B2022.12 | 69.4 | 2.1 | 62.8 | — | — | — | |
| DATAMODELSBackbone=OPT-13B2022.12 | 69.3 | 3.8 | 57.3 | — | — | — | |
| UN-CONDACCBackbone=OPT-13B2022.12 | 69 | 2.6 | 61.5 | — | — | — | |
| DARTModel Architecture=DEEPSEEK-R1-QWEN-8B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 68.65 | — | — | — | — | — | |
| ONESHOTBackbone=OPT-13B2022.12 | 68.3 | 2.3 | 62.7 | — | — | — | |
| UN-ONESHOTBackbone=OPT-13B2022.12 | 68 | 2.5 | 59.8 | — | — | — | |
| Best ModelBackbone LLM=Qwen 2.5 0.5B, Shot count=0-shot2025.10 | 67.2 | — | — | — | 5 | 1.4 | |
| DARTModel Architecture=LLAMA-3.1-8B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 66.21 | — | — | — | — | — | |
| UN-TOPPROMPTS-5Backbone=OPT-13B2022.12 | 66.2 | 3.4 | 54.6 | — | — | — | |
| DARTModel Architecture=QWEN3-14B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 65.69 | — | — | — | — | — | |
| ALL + CALIBBackbone=OPT-13B2022.12 | 65.5 | 4.9 | 51.8 | — | — | — | |
| TOPPROMPTS-10Backbone=OPT-13B2022.12 | 65.5 | 5.2 | 50.4 | — | — | — | |
| DATAMODELSBackbone=GPTJ-6B2022.12 | 65.2 | 0.9 | 63.4 | — | — | — | |
| ALLBackbone=OPT-13B2022.12 | 65.2 | 5.6 | 49.7 | — | — | — | |
| CONDACCBackbone=GPTJ-6B2022.12 | 65.1 | 1.6 | 61.1 | — | — | — | |
| UN-ALLBackbone=OPT-13B2022.12 | 64.8 | 5.3 | 49.3 | — | — | — | |
| RANDOMBackbone=OPT-13B2022.12 | 64.7 | 6.4 | 49.3 | — | — | — | |
| ONESHOTBackbone=GPTJ-6B2022.12 | 63.8 | 2.7 | 56.4 | — | — | — | |
| UN-CONDACCBackbone=GPTJ-6B2022.12 | 63.7 | 2.2 | 56 | — | — | — | |
| TOPPROMPTS-5Backbone=OPT-13B2022.12 | 63.5 | 6.3 | 51 | — | — | — | |
| UN-ONESHOTBackbone=GPTJ-6B2022.12 | 62.6 | 3.3 | 55.6 | — | — | — | |
| TOPPROMPTS-5Backbone=GPTJ-6B2022.12 | 62.3 | 3 | 54.3 | — | — | — | |
| ALL + CALIBBackbone=GPTJ-6B2022.12 | 61.2 | 3.9 | 50.4 | — | — | — | |
| TOPPROMPTS-10Backbone=GPTJ-6B2022.12 | 61.2 | 4 | 51.1 | — | — | — | |
| UN-TOPPROMPTS-5Backbone=GPTJ-6B2022.12 | 61.2 | 3.3 | 51.9 | — | — | — | |
| ALLBackbone=GPTJ-6B2022.12 | 61 | 3.8 | 49.7 | — | — | — | |
| UN-ALLBackbone=GPTJ-6B2022.12 | 60.8 | 3.5 | 49.6 | — | — | — | |
| RANDOMBackbone=GPTJ-6B2022.12 | 60 | 4.3 | 49.5 | — | — | — | |
| DARTModel Architecture=QWEN3-4B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 57.4 | — | — | — | — | — | |
| DARTModel Architecture=LLAMA-3.2-3B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 53.24 | — | — | — | — | — | |
| AioliModel Size=500M, Base Model=Qwen-2, Training Steps=2000, Batch Size=32, Context Length=512, K=20, E=102026.06 | — | — | — | 79.64 | — | — | |
| DoGEModel Size=500M, Base Model=Qwen-2, Training Steps=2000, Batch Size=32, Context Length=512, K=20, E=102026.06 | — | — | — | 81.04 | — | — | |
| DoReMiModel Size=500M, Base Model=Qwen-2, Training Steps=2000, Batch Size=32, Context Length=512, K=20, E=102026.06 | — | — | — | 77.43 | — | — | |
| Few-shot AccuracyMode=Few-shot2026.03 | — | — | — | 83.8 | — | — | |
| Linear ProbeTrain Dataset=ARC-C2026.03 | — | — | — | 55.48 | — | — | |
| Linear ProbeTrain Dataset=ARC-E2026.03 | — | — | — | 58.82 | — | — | |
| Linear ProbeTrain Dataset=OpenQA2026.03 | — | — | — | 56.79 | — | — | |
| Linear ProbeTrain Dataset=BoolQ2026.03 | — | — | — | 83.65 | — | — | |
| Linear ProbeTrain Dataset=Hellaswag2026.03 | — | — | — | 58.24 | — | — | |
| Linear ProbeTrain Dataset=ComQA2026.03 | — | — | — | 50.31 | — | — | |
| Linear ProbeTrain Dataset=CosQA2026.03 | — | — | — | 66.5 | — | — | |
| Linear ProbeTrain Dataset=SiQA2026.03 | — | — | — | 55.77 | — | — | |
| Skill-ItModel Size=500M, Base Model=Qwen-2, Training Steps=2000, Batch Size=32, Context Length=512, K=20, E=102026.06 | — | — | — | 80.34 | — | — | |
| TANDEMModel Size=500M, Base Model=Qwen-2, Training Steps=2000, Batch Size=32, Context Length=512, K=20, E=102026.06 | — | — | — | 80.04 | — | — | |
| TaTTrain Dataset=ARC-C2026.03 | — | — | — | 96.91 | — | — | |
| TaTTrain Dataset=ARC-E2026.03 | — | — | — | 78.56 | — | — | |
| TaTTrain Dataset=OpenQA2026.03 | — | — | — | 89.85 | — | — | |
| TaTTrain Dataset=BoolQ2026.03 | — | — | — | 85.05 | — | — | |
| TaTTrain Dataset=Hellaswag2026.03 | — | — | — | 64.22 | — | — | |
| TaTTrain Dataset=ComQA2026.03 | — | — | — | 68.72 | — | — | |
| TaTTrain Dataset=CosQA2026.03 | — | — | — | 77.77 | — | — | |
| TaTTrain Dataset=SiQA2026.03 | — | — | — | 63.85 | — | — | |
| UniformModel Size=500M, Base Model=Qwen-2, Training Steps=2000, Batch Size=32, Context Length=512, K=20, E=102026.06 | — | — | — | 80.29 | — | — | |
| Zero-shot AccuracyMode=Zero-shot2026.03 | — | — | — | 74.7 | — | — |