Mathematical Theorem Proving on MATH 500
91.2Proof-level AccuracyRaw CoT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Raw CoTLLM=DEEPSEEK-R1-0528-QWEN3-8B2026.06 | 91.2 | 3,544 | |
| CLSRLLM=QWEN3-32B2026.06 | 89.4 | 206 | |
| Raw CoTLLM=QWEN3-32B2026.06 | 89.3 | 845 | |
| CLSRLLM=DEEPSEEK-R1-0528-QWEN3-8B2026.06 | 88.1 | 1,753 | |
| Raw CoTLLM=QWEN3-8B2026.06 | 87.2 | 878 | |
| CLSRLLM=QWEN3-8B2026.06 | 86.8 | 257 | |
| SoTLLM=DEEPSEEK-R1-0528-QWEN3-8B2026.06 | 84.1 | 1,982 | |
| SoTLLM=QWEN3-32B2026.06 | 83.8 | 278 | |
| CCoTLLM=DEEPSEEK-R1-0528-QWEN3-8B2026.06 | 83.4 | 2,032 | |
| CCoTLLM=QWEN3-32B2026.06 | 82 | 382 | |
| CoDLLM=QWEN3-32B2026.06 | 81.5 | 365 | |
| SoTLLM=QWEN3-8B2026.06 | 81.3 | 294 | |
| CoDLLM=DEEPSEEK-R1-0528-QWEN3-8B2026.06 | 80.5 | 1,842 | |
| CCoTLLM=QWEN3-8B2026.06 | 78.4 | 423 | |
| CoDLLM=QWEN3-8B2026.06 | 74.5 | 389 | |
| Raw CoTLLM=LLaMA3-8B2026.06 | 51.6 | 704 | |
| CLSRLLM=LLaMA3-8B2026.06 | 48.2 | 257 | |
| CCoTLLM=LLaMA3-8B2026.06 | 45.6 | 315 | |
| SoTLLM=LLaMA3-8B2026.06 | 45.2 | 289 | |
| CoDLLM=LLaMA3-8B2026.06 | 42.2 | 280 | |
| BASESolver=Gemini 3 Pro, Evaluation Protocol=BASE, Prover=DeepSeek-Prover-V2, K=82026.06 | 31.4 | — | |
| BASESolver=R1-Qwen3-8B, Evaluation Protocol=BASE, Prover=DeepSeek-Prover-V2, K=82026.06 | 28.4 | — | |
| BASESolver=Qwen3-8B, Evaluation Protocol=BASE, Prover=DeepSeek-Prover-V2, K=82026.06 | 25.9 | — |