Science Question Answering on GPQA (Accuracy and Delta Avg)
91.5AccuracyCoT2-Meta
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CoT2-MetaBackbone=DeepSeek-V3.2, Strategy=Ours (CoT2-Meta)2026.03 | 91.5 | 10.5 | |
| CoT2-MetaBackbone=Claude-4.5, Strategy=Ours (CoT2-Meta)2026.03 | 90.4 | 14.5 | |
| Vanilla ToTBackbone=DeepSeek-V3.2, Strategy=Vanilla ToT2026.03 | 88.3 | 5.7 | |
| Best-of-16Backbone=DeepSeek-V3.2, Strategy=Best-of-162026.03 | 85.1 | 3 | |
| Vanilla ToTBackbone=Claude-4.5, Strategy=Vanilla ToT2026.03 | 83.5 | 8.3 | |
| Greedy CoTBackbone=DeepSeek-V3.2, Strategy=Greedy CoT2026.03 | 82.4 | — | |
| Best-of-16Backbone=Claude-4.5, Strategy=Best-of-162026.03 | 79.6 | 4.8 | |
| Greedy CoTBackbone=Claude-4.5, Strategy=Greedy CoT2026.03 | 74.2 | — | |
| CoT2-MetaBackbone=Qwen2.5-VL-7B, Strategy=Ours (CoT2-Meta)2026.03 | 59.8 | 12.2 | |
| Vanilla ToTBackbone=Qwen2.5-VL-7B, Strategy=Vanilla ToT2026.03 | 53.4 | 6.4 | |
| Best-of-16Backbone=Qwen2.5-VL-7B, Strategy=Best-of-162026.03 | 49.2 | 3.4 | |
| Greedy CoTBackbone=Qwen2.5-VL-7B, Strategy=Greedy CoT2026.03 | 44.7 | — | |
| DenseSparsity=0, Model Scale=72B, Model Family=Qwen 2.52025.10 | 42.63 | — | |
| ARMORSparsity=2:4+2.4%, Model Scale=72B, Model Family=Qwen 2.52025.10 | 40.4 | — | |
| ARMORSparsity=2:4+3.44%, Model Scale=32B, Model Family=Qwen 2.52025.10 | 39.51 | — | |
| DenseSparsity=0, Model Scale=32B, Model Family=Qwen 2.52025.10 | 38.84 | — | |
| DenseSparsity=0, Model Scale=14B, Model Family=Qwen 2.52025.10 | 38.17 | — | |
| Cog-DRIFTBase Model=Qwen3-4B-Instruct-25072026.04 | 36.01 | — | |
| WandaSparsity=2:4, Model Scale=32B, Model Family=Qwen 2.52025.10 | 35.27 | — | |
| ARMORSparsity=2:4+4.17%, Model Scale=14B, Model Family=Qwen 2.52025.10 | 33.48 | — | |
| DenseSparsity=0, Model Scale=7B, Model Family=Qwen 2.52025.10 | 33.03 | — | |
| SparseGPTSparsity=2:4, Model Scale=14B, Model Family=Qwen 2.52025.10 | 31.92 | — | |
| GRPOBase Model=Qwen3-4B-Instruct-25072026.04 | 31.81 | — | |
| ARMORSparsity=2:4+4.95%, Model Scale=7B, Model Family=Qwen 2.52025.10 | 31.47 | — | |
| NoWag-PSparsity=2:4, Model Scale=14B, Model Family=Qwen 2.52025.10 | 30.36 | — | |
| SparseGPTSparsity=2:4, Model Scale=32B, Model Family=Qwen 2.52025.10 | 30.36 | — | |
| WandaSparsity=2:4, Model Scale=14B, Model Family=Qwen 2.52025.10 | 29.91 | — | |
| SparseGPTSparsity=2:4, Model Scale=7B, Model Family=Qwen 2.52025.10 | 29.69 | — | |
| NoWag-PSparsity=2:4, Model Scale=72B, Model Family=Qwen 2.52025.10 | 28.35 | — | |
| SparseGPTSparsity=2:4, Model Scale=72B, Model Family=Qwen 2.52025.10 | 27.46 | — | |
| NoWag-PSparsity=2:4, Model Scale=7B, Model Family=Qwen 2.52025.10 | 27.23 | — | |
| NoWag-PSparsity=2:4, Model Scale=32B, Model Family=Qwen 2.52025.10 | 27.01 | — | |
| Few-shotBase Model=Qwen3-4B-Instruct-25072026.04 | 26.12 | — | |
| WandaSparsity=2:4, Model Scale=7B, Model Family=Qwen 2.52025.10 | 25.45 | — | |
| NuRL (Abstract)Base Model=Qwen3-4B-Instruct-25072026.04 | 24.72 | — | |
| WandaSparsity=2:4, Model Scale=72B, Model Family=Qwen 2.52025.10 | 23.88 | — | |
| Zero-shotBase Model=Qwen3-4B-Instruct-25072026.04 | 23.57 | — | |
| Cog-DRIFTBase Model=Llama3.2-3B-Instruct2026.04 | 23.34 | — | |
| RFTBase Model=Qwen3-4B-Instruct-25072026.04 | 23.04 | — | |
| NuRL (Prefix)Base Model=Qwen3-4B-Instruct-25072026.04 | 22.27 | — | |
| Few-shotBase Model=Llama3.2-3B-Instruct2026.04 | 17.89 | — | |
| NuRL (Abstract)Base Model=Llama3.2-3B-Instruct2026.04 | 16.98 | — | |
| RFTBase Model=Llama3.2-3B-Instruct2026.04 | 16.13 | — | |
| NuRL (Prefix)Base Model=Llama3.2-3B-Instruct2026.04 | 15.79 | — | |
| Zero-shotBase Model=Llama3.2-3B-Instruct2026.04 | 15.32 | — | |
| GRPOBase Model=Llama3.2-3B-Instruct2026.04 | 12.03 | — |