Reasoning on AIME 24 (Traces, Consistency, Accuracy)
70AccuracyPETS-On.
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PETS-On.LLM=GPT-20B, Confidence-Weighted=false2026.02 | 70 | 130 | 100 | |
| PETS-On.LLM=GPT-120B, Confidence-Weighted=false2026.02 | 69.8 | 107 | 100 | |
| PETS-On. (conf)LLM=GPT-120B, Confidence-Weighted=true2026.02 | 69.5 | 92 | 100 | |
| PETS-On. (conf)LLM=GPT-20B, Confidence-Weighted=true2026.02 | 68.8 | 104 | 99 | |
| Uniform (conf)LLM=GPT-20B, Confidence-Weighted=true2026.02 | 68 | 171 | 98 | |
| UniformLLM=GPT-20B, Confidence-Weighted=false2026.02 | 67.7 | 237 | 98 | |
| Uniform (conf)LLM=GPT-120B, Confidence-Weighted=true2026.02 | 67.7 | 144 | 98 | |
| UniformLLM=GPT-120B, Confidence-Weighted=false2026.02 | 67 | 205 | 97 | |
| PETS-On.LLM=Qwen3-30B, Confidence-Weighted=false2026.02 | 65 | 81 | 100 | |
| PETS-On.LLM=Qwen-Long, Confidence-Weighted=false2026.02 | 65 | 83 | 100 | |
| UniformLLM=Qwen-Long, Confidence-Weighted=false2026.02 | 65 | 82 | 100 | |
| PETS-On. (conf)LLM=Qwen3-30B, Confidence-Weighted=true2026.02 | 64.8 | 81 | 100 | |
| PETS-On. (conf)LLM=Qwen-Long, Confidence-Weighted=true2026.02 | 64.8 | 86 | 99 | |
| Uniform (conf)LLM=Qwen-Long, Confidence-Weighted=true2026.02 | 64.8 | 101 | 99 | |
| UniformLLM=Qwen3-30B, Confidence-Weighted=false2026.02 | 64.7 | 83 | 100 | |
| Uniform (conf)LLM=Qwen3-30B, Confidence-Weighted=true2026.02 | 64.7 | 83 | 100 | |
| PETS-On. (conf)LLM=Qwen3-4B, Confidence-Weighted=true2026.02 | 60 | 185 | 96 | |
| Uniform (conf)LLM=Qwen3-4B, Confidence-Weighted=true2026.02 | 58.8 | 405 | 94 | |
| UniformLLM=Qwen3-4B, Confidence-Weighted=false2026.02 | 56.8 | 393 | 94 | |
| PETS-On.LLM=Qwen3-4B, Confidence-Weighted=false2026.02 | 56.2 | 170 | 100 | |
| MAS-ZEROLLM Backbone=Llama3.3-70B2025.05 | 37.5 | — | — | |
| MAS-ZEROLLM Backbone=GPT-4o2025.05 | 33.33 | — | — | |
| ReConcileLLM Backbone=Llama3.3-70B2025.05 | 33.33 | — | — | |
| MaASLLM Backbone=Llama3.3-70B2025.05 | 33.33 | — | — | |
| AFlowLLM Backbone=Llama3.3-70B2025.05 | 33.33 | — | — | |
| AFlowLLM Backbone=Qwen2.5-32B2025.05 | 33.33 | — | — | |
| CoT-SCLLM Backbone=Llama3.3-70B2025.05 | 29.17 | — | — | |
| Self-RefineLLM Backbone=Llama3.3-70B2025.05 | 29.17 | — | — | |
| MADLLM Backbone=Llama3.3-70B2025.05 | 29.17 | — | — | |
| DyLANLLM Backbone=Llama3.3-70B2025.05 | 29.17 | — | — | |
| MAS-ZEROLLM Backbone=Qwen2.5-32B2025.05 | 29.17 | — | — | |
| SPPLLM Backbone=Llama3.3-70B2025.05 | 26.39 | — | — | |
| MAS-GPTLLM Backbone=Llama3.3-70B2025.05 | 26.39 | — | — | |
| MAS-GPTLLM Backbone=Qwen2.5-32B2025.05 | 23.61 | — | — | |
| AFlowLLM Backbone=GPT-4o2025.05 | 20.83 | — | — | |
| DebateLLM Backbone=Llama3.3-70B2025.05 | 20.83 | — | — | |
| DyLANLLM Backbone=Qwen2.5-32B2025.05 | 20.83 | — | — | |
| MaASLLM Backbone=Qwen2.5-32B2025.05 | 20.83 | — | — | |
| SPPLLM Backbone=Qwen2.5-32B2025.05 | 19.44 | — | — | |
| CoT-SCLLM Backbone=GPT-4o2025.05 | 16.67 | — | — | |
| CoTLLM Backbone=Llama3.3-70B2025.05 | 16.67 | — | — | |
| CoT-SCLLM Backbone=Qwen2.5-32B2025.05 | 16.67 | — | — | |
| Self-RefineLLM Backbone=Qwen2.5-32B2025.05 | 16.67 | — | — | |
| MADLLM Backbone=Qwen2.5-32B2025.05 | 16.67 | — | — | |
| MADLLM Backbone=GPT-4o2025.05 | 13.89 | — | — | |
| MAS-GPTLLM Backbone=GPT-4o2025.05 | 13.89 | — | — | |
| ReConcileLLM Backbone=GPT-4o2025.05 | 12.5 | — | — | |
| MaASLLM Backbone=GPT-4o2025.05 | 12.5 | — | — | |
| CoTLLM Backbone=Qwen2.5-32B2025.05 | 12.5 | — | — | |
| ReConcileLLM Backbone=Qwen2.5-32B2025.05 | 12.5 | — | — | |
| ADASLLM Backbone=Qwen2.5-32B2025.05 | 12.5 | — | — | |
| DyLANLLM Backbone=GPT-4o2025.05 | 11.11 | — | — | |
| SPPLLM Backbone=GPT-4o2025.05 | 9.72 | — | — | |
| CoTLLM Backbone=GPT-4o2025.05 | 8.33 | — | — | |
| DebateLLM Backbone=Qwen2.5-32B2025.05 | 8.33 | — | — | |
| ADASLLM Backbone=Llama3.3-70B2025.05 | 8.3 | — | — | |
| DebateLLM Backbone=GPT-4o2025.05 | 4.17 | — | — | |
| Self-RefineLLM Backbone=GPT-4o2025.05 | 4.17 | — | — |