Reasoning on LiveCodeBench (pass@1 accuracy)
55.45pass@1 AccuracyBaseline
Evaluation Results
| Method | Links | |
|---|---|---|
| BaselineModel=QwQ-32B, Bit-Width=16-162025.10 | 55.45 | |
| ThinKV (k=1024)Model=QwQ-32B, Bit-Width=3.4-3.42025.10 | 50.47 | |
| BF16 BaselineModel Backbone=Qwen3, Model Size=32B2026.06 | 49.9 | |
| GraphPOModel=Deepseek-R1-Distill-Qwen-7B2026.06 | 48.3 | |
| BaselineModel=R1-Qwen-14B, Bit-Width=16-162025.10 | 47.9 | |
| MR-GPTQModel Backbone=Qwen3, Model Size=14B2026.06 | 47.2 | |
| RTNModel Backbone=Qwen3, Model Size=14B2026.06 | 46.7 | |
| ReSETModel Backbone=Qwen3, Model Size=32B2026.06 | 46.7 | |
| PM-KVQModel=QwQ-32B, Bit-Width=3.5-3.52025.10 | 46.68 | |
| BF16 BaselineModel Backbone=Qwen3, Model Size=14B2026.06 | 46.5 | |
| RTNModel Backbone=Qwen3, Model Size=32B2026.06 | 46.5 | |
| ReSETModel Backbone=Qwen3, Model Size=14B2026.06 | 46.1 | |
| ThinKV (k=1024)Model=R1-Qwen-14B, Bit-Width=3.5-3.52025.10 | 45.84 | |
| 4/6Model Backbone=Qwen3, Model Size=32B2026.06 | 45.7 | |
| 4/6Model Backbone=Qwen3, Model Size=14B2026.06 | 45.3 | |
| BRQModel Backbone=Qwen3, Model Size=32B2026.06 | 45.1 | |
| MR-GPTQModel Backbone=Qwen3, Model Size=32B2026.06 | 45.1 | |
| PROSModel=Deepseek-R1-Distill-Qwen-7B2026.06 | 44.8 | |
| SPOModel=Deepseek-R1-Distill-Qwen-7B2026.06 | 44.7 | |
| TREE-GRPOModel=Deepseek-R1-Distill-Qwen-7B2026.06 | 44.1 | |
| TreePOModel=Deepseek-R1-Distill-Qwen-7B2026.06 | 43.5 | |
| BF16 BaselineModel Backbone=Qwen3, Model Size=8B2026.06 | 43.2 | |
| MR-GPTQModel Backbone=Qwen3, Model Size=8B2026.06 | 42.9 | |
| TreeRLModel=Deepseek-R1-Distill-Qwen-7B2026.06 | 42.5 | |
| ReSETModel Backbone=Qwen3, Model Size=8B2026.06 | 42.1 | |
| PM-KVQModel=R1-Qwen-14B, Bit-Width=3.2-3.22025.10 | 41.97 | |
| DAPOModel=Deepseek-R1-Distill-Qwen-7B2026.06 | 41.4 | |
| BF16 BaselineModel Backbone=All Models, Model Size=Avg2026.06 | 41.3 | |
| KIVIModel=QwQ-32B, Bit-Width=2-22025.10 | 40.75 | |
| GRPOModel=Deepseek-R1-Distill-Qwen-7B2026.06 | 40.7 | |
| ReSETModel Backbone=All Models, Model Size=Avg2026.06 | 40.2 | |
| MR-GPTQModel Backbone=All Models, Model Size=Avg2026.06 | 39.5 | |
| RTNModel Backbone=All Models, Model Size=Avg2026.06 | 39.2 | |
| BRQModel Backbone=Qwen3, Model Size=14B2026.06 | 39 | |
| BF16 BaselineModel Backbone=R1-Qwen, Model Size=14B2026.06 | 38.5 | |
| 4/6Model Backbone=All Models, Model Size=Avg2026.06 | 38 | |
| ReSETModel Backbone=R1-Qwen, Model Size=14B2026.06 | 37.9 | |
| BRQModel Backbone=R1-Qwen, Model Size=14B2026.06 | 37.7 | |
| RTNModel Backbone=R1-Qwen, Model Size=14B2026.06 | 37.1 | |
| BRQModel Backbone=All Models, Model Size=Avg2026.06 | 37.1 | |
| BaseModel=Deepseek-R1-Distill-Qwen-7B2026.06 | 36.8 | |
| RTNModel Backbone=Qwen3, Model Size=8B2026.06 | 36.4 | |
| 4/6Model Backbone=R1-Qwen, Model Size=14B2026.06 | 36.4 | |
| BRQModel Backbone=Qwen3, Model Size=8B2026.06 | 36.2 | |
| 4/6Model Backbone=Qwen3, Model Size=8B2026.06 | 35.8 | |
| MR-GPTQModel Backbone=R1-Qwen, Model Size=14B2026.06 | 34.9 | |
| KIVIModel=R1-Qwen-14B, Bit-Width=2-22025.10 | 34.56 | |
| GraphPOModel=Qwen3-8B-Base2026.06 | 34.5 | |
| PROSModel=Qwen3-8B-Base2026.06 | 31.1 | |
| SPOModel=Qwen3-8B-Base2026.06 | 30.8 | |
| TREE-GRPOModel=Qwen3-8B-Base2026.06 | 30.4 | |
| RTNModel Backbone=R1-Qwen, Model Size=7B2026.06 | 29.5 | |
| TreePOModel=Qwen3-8B-Base2026.06 | 29.4 | |
| ReSETModel Backbone=R1-Qwen, Model Size=7B2026.06 | 28.4 | |
| BF16 BaselineModel Backbone=R1-Qwen, Model Size=7B2026.06 | 28.2 | |
| TreeRLModel=Qwen3-8B-Base2026.06 | 28.1 | |
| BRQModel Backbone=R1-Qwen, Model Size=7B2026.06 | 27.6 | |
| DAPOModel=Qwen3-8B-Base2026.06 | 27.6 | |
| MR-GPTQModel Backbone=R1-Qwen, Model Size=7B2026.06 | 27.2 | |
| GRPOModel=Qwen3-8B-Base2026.06 | 27.2 | |
| 4/6Model Backbone=R1-Qwen, Model Size=7B2026.06 | 27 | |
| BaseModel=Qwen3-8B-Base2026.06 | 22.9 | |
| GraphPOModel=Qwen2.5-7B-Math2026.06 | 16.9 | |
| SPOModel=Qwen2.5-7B-Math2026.06 | 13.6 | |
| PROSModel=Qwen2.5-7B-Math2026.06 | 13.5 | |
| TREE-GRPOModel=Qwen2.5-7B-Math2026.06 | 13.4 | |
| TreePOModel=Qwen2.5-7B-Math2026.06 | 13 | |
| DAPOModel=Qwen2.5-7B-Math2026.06 | 12.6 | |
| TreeRLModel=Qwen2.5-7B-Math2026.06 | 12.5 | |
| GRPOModel=Qwen2.5-7B-Math2026.06 | 11.4 | |
| BaseModel=Qwen2.5-7B-Math2026.06 | 5.7 |