Commonsense Reasoning on OpenBookQA
91.2AccuracySingle
Evaluation results
| Method | Links | |
|---|---|---|
| SingleBackbone=DeepSeek-R1-Distill-Qwen-32B2026.04 | 91.2 | |
| TextMASBackbone=DeepSeek-R1-Distill-Qwen-32B2026.04 | 91.2 | |
| DiffMASBackbone=DeepSeek-R1-Distill-Qwen-32B2026.04 | 91.2 | |
| Fine-tuning2023.06 | 91 | |
| TTVSBackbone=Qwen3-4B2026.04 | 90.7 | |
| LatentMASBackbone=DeepSeek-R1-Distill-Qwen-32B2026.04 | 89.6 | |
| DiffMASBackbone=Qwen3-14B2026.04 | 89 | |
| LatentMASBackbone=Qwen3-14B2026.04 | 88.8 | |
| BaLoRAModel=Llama-3-8B, Params (%)=0.71452026.04 | 88.4 | |
| HiRAModel=Llama-3-8B, Params (%)=0.70022026.04 | 88.32 | |
| TTRLBackbone=Qwen3-4B2026.04 | 87.5 | |
| CoK + SC + F2-VBase Model=gpt-3.5-turbo2023.06 | 87 | |
| SingleBackbone=Qwen3-14B2026.04 | 86.6 | |
| TextMASBackbone=Qwen3-14B2026.04 | 86.2 | |
| CoK + SCBase Model=gpt-3.5-turbo2023.06 | 86.1 | |
| LatentMASModel=Qwen3-8B, Communication Category=Training-free Latent Communication2026.04 | 85.8 | |
| C2CModel=Qwen3-8B, Communication Category=Trained Latent Communication2026.04 | 85.8 | |
| DiffMASModel=Qwen3-8B, Communication Category=Trained Latent Communication2026.04 | 85.8 | |
| DoRAModel=Llama-3-8B, Params (%)=0.70022026.04 | 85.8 | |
| MoRAModel=Llama-3-8B, Params (%)=0.69972026.04 | 85.6 | |
| BaLoRAModel=Llama-2-7B, Params (%)=0.84272026.04 | 85.4 | |
| Manual CoT + SCBase Model=gpt-3.5-turbo2023.06 | 85 | |
| CoK + F2-VBase Model=gpt-3.5-turbo2023.06 | 85 | |
| HiRAModel=Llama-2-7B, Params (%)=0.82562026.04 | 84.6 | |
| TextMASModel=Qwen3-8B, Communication Category=Text Communication2026.04 | 84.2 | |
| SingleModel=Qwen3-8B, Communication Category=Text Communication2026.04 | 83.6 | |
| CoKBase Model=gpt-3.5-turbo2023.06 | 83.5 | |
| DiffMASModel=Qwen3-4B, Communication Category=Trained Latent Communication2026.04 | 83.2 | |
| Manual CoTBase Model=gpt-3.5-turbo2023.06 | 82.6 | |
| InitBackbone=Qwen3-4B2026.04 | 82.6 | |
| DoRAModel=Llama-2-7B, Params (%)=0.82562026.04 | 82.4 | |
| LatentMASBackbone=Mistral3-8B2026.04 | 82.2 | |
| DiffMASBackbone=Mistral3-8B2026.04 | 82.2 | |
| TextMASModel=Qwen3-4B, Communication Category=Text Communication2026.04 | 81.8 | |
| TTVSBackbone=Qwen2.5-Instruct-3B2026.04 | 81.2 | |
| MoRAModel=Llama-2-7B, Params (%)=0.82412026.04 | 81.2 | |
| SingleBackbone=Mistral3-8B2026.04 | 81 | |
| LoRAModel=Llama-2-7B, Params (%)=0.82562026.04 | 81 | |
| TextMASBackbone=Mistral3-8B2026.04 | 80.4 | |
| SingleModel=Qwen3-4B, Communication Category=Text Communication2026.04 | 80.1 | |
| C2CModel=Qwen3-4B, Communication Category=Trained Latent Communication2026.04 | 79.4 | |
| LoRAModel=Llama-3-8B, Params (%)=0.70022026.04 | 79 | |
| TTRLBackbone=Qwen2.5-Instruct-3B2026.04 | 78.9 | |
| LatentMASModel=Qwen3-4B, Communication Category=Training-free Latent Communication2026.04 | 77.6 | |
| InitBackbone=Qwen2.5-Instruct-3B2026.04 | 77.2 | |
| Few-Shot SPBase Model=text-davinci-0022023.06 | 76.6 | |
| CoK + F2-VBase Model=text-davinci-0022023.06 | 74.8 | |
| ChatGPTModel=ChatGPT2026.04 | 74.8 | |
| CoKBase Model=text-davinci-0022023.06 | 73.9 | |
| TTVSBackbone=LLaMA-3.2-3B-Instruct2026.04 | 73.3 | |
| Manual CoTBase Model=text-davinci-0022023.06 | 73 | |
| TTRLBackbone=LLaMA-3.2-3B-Instruct2026.04 | 71.9 | |
| Zero-Shot CoTBase Model=text-davinci-0022023.06 | 68.4 | |
| SnapKVBackbone=Nemotron-Nano-8B, Budget=1282025.12 | 68 | |
| SnapKVBackbone=Nemotron-Nano-8B, Budget=3842025.12 | 66 | |
| SnapKVBackbone=Nemotron-Nano-8B, Budget=5122025.12 | 66 | |
| InitBackbone=LLaMA-3.2-3B-Instruct2026.04 | 65.5 | |
| FullBackbone=Nemotron-Nano-8B2025.12 | 64 | |
| SnapKVBackbone=Nemotron-Nano-8B, Budget=2562025.12 | 63 | |
| H2OBackbone=Nemotron-Nano-8B, Budget=5122025.12 | 62 | |
| StreamingLLMBackbone=Nemotron-Nano-8B, Budget=5122025.12 | 62 | |
| H2OBackbone=Nemotron-Nano-8B, Budget=1282025.12 | 59 | |
| H2OBackbone=Nemotron-Nano-8B, Budget=2562025.12 | 59 | |
| H2OBackbone=Nemotron-Nano-8B, Budget=3842025.12 | 58 | |
| KnormBackbone=Nemotron-Nano-8B, Budget=5122025.12 | 57 | |
| StreamingLLMBackbone=Nemotron-Nano-8B, Budget=3842025.12 | 52 | |
| RKVBackbone=Nemotron-Nano-8B, Budget=3842025.12 | 51 | |
| RKVBackbone=Nemotron-Nano-8B, Budget=5122025.12 | 51 | |
| KnormBackbone=Nemotron-Nano-8B, Budget=3842025.12 | 48 | |
| StreamingLLMBackbone=Nemotron-Nano-8B, Budget=2562025.12 | 46 | |
| RFID-MoEBackbone=Qwen2-57B-A14B, Ratio=40%, Evaluation Protocol=Zero-shot2026.02 | 46 | |
| RFID-MoEBackbone=Qwen3-30B-A3B-2507, Ratio=20%, Evaluation Protocol=Zero-shot2026.02 | 45 | |
| Zero-Shot SPBase Model=text-davinci-0022023.06 | 44.7 | |
| KnormBackbone=Nemotron-Nano-8B, Budget=2562025.12 | 44 | |
| RKVBackbone=Nemotron-Nano-8B, Budget=2562025.12 | 44 | |
| RFID-MoEBackbone=DeepSeekMoE-16B-Base, Ratio=40%, Evaluation Protocol=Zero-shot2026.02 | 43 | |
| MLRA-4Evaluation Mode=Zero-shot2026.03 | 43 | |
| MLAEvaluation Mode=Zero-shot2026.03 | 42.6 | |
| GTAEvaluation Mode=Zero-shot2026.03 | 42.6 | |
| MFAEvaluation Mode=Zero-shot2026.03 | 42.4 | |
| RFID-MoEBackbone=Qwen1.5-MoE-A2.7B, Ratio=40%, Evaluation Protocol=Zero-shot2026.02 | 42 | |
| GQAEvaluation Mode=Zero-shot2026.03 | 42 | |
| MLRA-2Evaluation Mode=Zero-shot2026.03 | 42 | |
| MQAEvaluation Mode=Zero-shot2026.03 | 41.8 | |
| TPAEvaluation Mode=Zero-shot2026.03 | 41.6 | |
| GLA-4Evaluation Mode=Zero-shot2026.03 | 41.2 | |
| MHAEvaluation Mode=Zero-shot2026.03 | 40.8 | |
| GLA-2Evaluation Mode=Zero-shot2026.03 | 40.6 | |
| HADESNumber of Parameters=1B, Zero-shot=true2026.03 | 38.6 | |
| DeltaNetNumber of Parameters=1.3B, Zero-shot=true2026.03 | 38.4 | |
| TinyLlama-v1.0Mode=Zero-shot, Conversion=Original (Baseline)2025.11 | 38.2 | |
| Mamba2Number of Parameters=1.3B, Zero-shot=true2026.03 | 37.4 | |
| RetNetNumber of Parameters=1.3B, Zero-shot=true2026.03 | 36.6 | |
| StreamingLLMBackbone=Nemotron-Nano-8B, Budget=1282025.12 | 36 | |
| TinyLlama-v1.0 (Midpoint)Mode=Zero-shot, Conversion=Midpoint Discretization2025.11 | 35.6 | |
| RKVBackbone=Nemotron-Nano-8B, Budget=1282025.12 | 35 | |
| KnormBackbone=Nemotron-Nano-8B, Budget=1282025.12 | 32 | |
| ShadowKVBackbone=Nemotron-Nano-8B2025.12 | 31 | |
| DeltaNetZero-shot=true, Parameters=370M2026.03 | 29.6 | |
| Prompt TuningModel=Llama-3-8B, Params (%)=0.00102026.04 | 29.2 |