Code Reasoning on SuperGPQA Code SGPQA-1k (Accuracy, Pass@1)
47.4AccuracyDFT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DFTModel=Qwen3 32B2025.10 | 47.4 | 47.4 | |
| RandomModel=Qwen3 32B2025.10 | 45.5 | 45.5 | |
| OriginalModel=Qwen3 32B2025.10 | 43.9 | 43.9 | |
| VCOREModel=Qwen3 32B2025.10 | 43.1 | 43.1 | |
| iw-SFTModel=Qwen3 32B2025.10 | 42.4 | 42.4 | |
| SFTModel=Qwen3 32B2025.10 | 40.6 | 40.6 | |
| SFTModel=Qwen3 8B2025.10 | 37.9 | 37.9 | |
| DFTModel=Qwen3 8B2025.10 | 37.5 | 37.5 | |
| iw-SFTModel=Qwen3 8B2025.10 | 35.9 | 35.9 | |
| RandomModel=Qwen3 8B2025.10 | 35.9 | 35.9 | |
| VCOREModel=Qwen3 8B2025.10 | 35.4 | 35.4 | |
| OriginalModel=Qwen3 8B2025.10 | 34.3 | 34.3 | |
| DFTModel=Qwen3 4B2025.10 | 33.4 | 33.4 | |
| iw-SFTModel=Qwen3 4B2025.10 | 32.6 | 32.6 | |
| VCOREModel=Qwen3 4B2025.10 | 32.3 | 32.3 | |
| SFTModel=Qwen3 4B2025.10 | 31.8 | 31.8 | |
| RandomModel=Qwen3 4B2025.10 | 31.3 | 31.3 | |
| OriginalModel=Qwen3 4B2025.10 | 29.1 | 29.1 | |
| OriginalModel=LLaMA3.1 8B Instruct2025.10 | 18.6 | 18.6 | |
| VCOREModel=LLaMA3.1 8B Instruct2025.10 | 15.8 | 15.8 | |
| SFTModel=LLaMA3.1 8B Instruct2025.10 | 9.5 | 9.5 | |
| RandomModel=LLaMA3.1 8B Instruct2025.10 | 8.5 | 8.5 | |
| iw-SFTModel=LLaMA3.1 8B Instruct2025.10 | 6.6 | 6.6 | |
| DFTModel=LLaMA3.1 8B Instruct2025.10 | 5.1 | 5.1 |