Reasoning on GPQA (pass@1)
90.1Pass@1DeepSeek-V4-pro
Evaluation results
| Method | Links | |
|---|---|---|
| DeepSeek-V4-proModel Scale=Reference, Model Category=Open-source frontier2026.06 | 90.1 | |
| Qwen3-maxModel Scale=Reference, Model Category=Closed-source frontier2026.06 | 85 | |
| GPT-5(high)Model Scale=Reference, Model Category=Closed-source frontier2026.06 | 84.8 | |
| GPT-o1-previewModel Scale=Reference, Model Category=Closed-source frontier2026.06 | 73.3 | |
| DeepSeek-R1Model Scale=Reference, Model Category=Open-source frontier2026.06 | 71.2 | |
| DiScOModel Scale=32B, Model Category=Ours2026.06 | 64 | |
| QwQ-32B-previewModel Scale=32B, Model Category=Reasoning-specialized models2026.06 | 62.4 | |
| DeepSeek-R1-Distill-Qwen-32BModel Scale=32B, Model Category=Reasoning-specialized models2026.06 | 62.1 | |
| Qwen3-32BModel Scale=32B, Model Category=Reasoning-specialized models2026.06 | 62.1 | |
| ReasonFlux-32BModel Scale=32B, Model Category=Reasoning-specialized models2026.06 | 61.2 | |
| DiScOw/o ITDEModel Scale=32B, Model Category=Ours2026.06 | 60.5 | |
| GPT-o1-miniModel Scale=Reference, Model Category=Closed-source frontier2026.06 | 60 | |
| Claude3.5-SonnetModel Scale=Reference, Model Category=Closed-source frontier2026.06 | 59.4 | |
| FP16Model=Qwen3-8B, Speedup=1.0×2026.05 | 58.78 | |
| R-QATModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.45×2026.05 | 58.49 | |
| GPTQModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.45×2026.05 | 57.89 | |
| DeepSeek-V3Model Scale=Reference, Model Category=Open-source frontier2026.06 | 57.8 | |
| LAQuantModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.45×2026.05 | 57.23 | |
| ParoQModel=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.32×2026.05 | 56.85 | |
| Sky-T1-32B-previewModel Scale=32B, Model Category=Reasoning-specialized models2026.06 | 56.8 | |
| ParoQ++Model=Qwen3-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.32×2026.05 | 56.66 | |
| FP16Model=Qwen3-4B, Speedup=1.0×2026.05 | 55.59 | |
| LAQuantModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.0×2026.05 | 53.6 | |
| GPT-4oModel Scale=Reference, Model Category=Closed-source frontier2026.06 | 53.6 | |
| ParoQ++Model=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.84×2026.05 | 53.44 | |
| ParoQModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.84×2026.05 | 52.46 | |
| R-QATModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.0×2026.05 | 52.15 | |
| GPTQModel=Qwen3-4B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.0×2026.05 | 51.61 | |
| Qwen2.5-32B-InstructModel Scale=32B, Model Category=Instruction-tuned models2026.06 | 48.8 | |
| LLaMA3.1-405B-InstructModel Scale=Reference, Model Category=Open-source frontier2026.06 | 48.2 | |
| FP16Model=R1-Distill-Llama-8B, Speedup=1.0×2026.05 | 48.17 | |
| InitialModel Size=8B, Sampling trajectories=82026.01 | 48.11 | |
| LLaMA3.1-70B-InstructModel Scale=Reference, Model Category=Open-source frontier2026.06 | 47.6 | |
| SafePathModel Size=8B, Sampling trajectories=82026.01 | 47.41 | |
| SafeKeyModel Size=7B, Sampling trajectories=82026.01 | 47.29 | |
| SafeChainModel Size=7B, Sampling trajectories=82026.01 | 46.78 | |
| InitialModel Size=7B, Sampling trajectories=82026.01 | 46.65 | |
| GPTQModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.46×2026.05 | 46.65 | |
| ParoQ++Model=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.33×2026.05 | 46.46 | |
| ParoQModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.33×2026.05 | 46.34 | |
| THINKSAFEModel Size=8B, Sampling trajectories=82026.01 | 46.28 | |
| SafePathModel Size=7B, Sampling trajectories=82026.01 | 46.15 | |
| R-QATModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.46×2026.05 | 46.09 | |
| STAR-1Model Size=7B, Sampling trajectories=82026.01 | 46.02 | |
| LAQuantModel=R1-Distill-Llama-8B, Quantization Bit-width=4-bit, Group Size=128, Speedup=2.46×2026.05 | 45.55 | |
| THINKSAFEModel Size=7B, Sampling trajectories=82026.01 | 45.2 | |
| DiScOModel Scale=7B, Model Category=Ours2026.06 | 45 | |
| DirectRefusalModel Size=7B, Sampling trajectories=82026.01 | 44.95 | |
| STAR-1Model Size=8B, Sampling trajectories=82026.01 | 43.69 | |
| DirectRefusalModel Size=8B, Sampling trajectories=82026.01 | 43.5 | |
| DiScOw/o ITDEModel Scale=7B, Model Category=Ours2026.06 | 43.4 | |
| DeepSeek-Coder-V2-InstructModel Scale=Reference, Model Category=Open-source frontier2026.06 | 43 | |
| SafeKeyModel Size=8B, Sampling trajectories=82026.01 | 42.49 | |
| Qwen2.5-Math-72B-InstructModel Scale=Reference, Model Category=Open-source frontier2026.06 | 42.1 | |
| SafeChainModel Size=8B, Sampling trajectories=82026.01 | 42.05 | |
| Qwen3-8BModel Scale=7B, Model Category=Reasoning-specialized models2026.06 | 41.9 | |
| rStar-Math-7BModel Scale=7B, Model Category=Reasoning-specialized models2026.06 | 41.8 | |
| FP16Model=Qwen3-1.7B, Speedup=1.0×2026.05 | 40.7 | |
| Qwen2.5-Math-7B-InstructModel Scale=7B, Model Category=Math instruction-tuned models2026.06 | 40.6 | |
| DIVERModel Scale=7B, Model Category=Diversity-oriented RL models2026.06 | 40.1 | |
| DiRLBackbone=Qwen3-4B2026.06 | 39.9 | |
| Entropy-RLModel Scale=7B, Model Category=Diversity-oriented RL models2026.06 | 37.9 | |
| Pass@k TrainingModel Scale=7B, Model Category=Diversity-oriented RL models2026.06 | 36.8 | |
| LAQuantModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.64×2026.05 | 36.43 | |
| ParoQ++Model=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.48×2026.05 | 36.3 | |
| ReasonFlux-7BModel Scale=7B, Model Category=Reasoning-specialized models2026.06 | 35.9 | |
| G2RLBackbone=Qwen3-4B2026.06 | 35.9 | |
| ParoQModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.48×2026.05 | 35.86 | |
| R-QATModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.64×2026.05 | 35.7 | |
| GRPO w/ Clip-higherModel Scale=7B, Model Category=Diversity-oriented RL models2026.06 | 35.2 | |
| NuminaMath-72B-CoTModel Scale=Reference, Model Category=Open-source frontier2026.06 | 34.4 | |
| EVOL-RLBackbone=Qwen3-4B2026.06 | 33.8 | |
| GPTQModel=Qwen3-1.7B, Quantization Bit-width=4-bit, Group Size=128, Speedup=1.64×2026.05 | 33.14 | |
| Entropy BonusBackbone=Qwen3-4B2026.06 | 32.8 | |
| DirectRefusalModel Size=1.5B, Sampling trajectories=82026.01 | 32.7 | |
| SafePathModel Size=1.5B, Sampling trajectories=82026.01 | 32.51 | |
| GRPOBackbone=Qwen3-4B2026.06 | 32.3 | |
| InitialModel Size=1.5B, Sampling trajectories=82026.01 | 31.94 | |
| STAR-1Model Size=1.5B, Sampling trajectories=82026.01 | 31.25 | |
| THINKSAFEModel Size=1.5B, Sampling trajectories=82026.01 | 31.19 | |
| SafeKeyModel Size=1.5B, Sampling trajectories=82026.01 | 28.72 | |
| SafeChainModel Size=1.5B, Sampling trajectories=82026.01 | 28.28 | |
| Qwen2.5-Math-7BModel Scale=7B, Model Category=Math instruction-tuned models2026.06 | 27.7 | |
| DiRLBackbone=Qwen3-1.7B2026.06 | 25.8 | |
| G2RLBackbone=Qwen3-1.7B2026.06 | 23.2 | |
| GRPOBackbone=Qwen3-1.7B2026.06 | 22.7 | |
| EVOL-RLBackbone=Qwen3-1.7B2026.06 | 22.7 | |
| Entropy BonusBackbone=Qwen3-1.7B2026.06 | 22.5 | |
| SuperCorrect-7BModel Scale=7B, Model Category=Reasoning-specialized models2026.06 | 20.7 | |
| DeepSeek-R1-Distill-Qwen-7BModel Scale=7B, Model Category=Reasoning-specialized models2026.06 | 10.6 | |
| LLaMA3.1-8B-InstructModel Scale=7B, Model Category=General instruction-tuned models2026.06 | 7.6 | |
| Mathstral-7B-v0.1Model Scale=7B, Model Category=General instruction-tuned models2026.06 | 7.1 |