Question Answering on GPQA Diamond (Accuracy and BF Cost)
87.35AccuracyCoT-SC
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CoT-SCLLM Backbone=GPT-52026.06 | 87.35 | 46.39 | |
| DyLANLLM Backbone=Gemini-2.5-Pro2026.06 | 87.35 | 57.46 | |
| CoTLLM Backbone=GPT-52026.06 | 87.14 | 7.24 | |
| MaASLLM Backbone=GPT-52026.06 | 86.95 | 38.5 | |
| MaASLLM Backbone=Gemini-2.5-Pro2026.06 | 86.95 | 94.83 | |
| MAS-ZeroLLM Backbone=GPT-52026.06 | 86.75 | 636.8 | |
| MAS-ZeroLLM Backbone=Gemini-2.5-Pro2026.06 | 86.14 | 526 | |
| MAS-OrchestraLLM Backbone=GPT-52026.06 | 85.54 | 41.7 | |
| ADASLLM Backbone=GPT-52026.06 | 85.23 | 832.1 | |
| ADASLLM Backbone=Gemini-2.5-Pro2026.06 | 84.52 | 694 | |
| AFlowLLM Backbone=GPT-52026.06 | 84.13 | 274.6 | |
| AFlowLLM Backbone=Gemini-2.5-Pro2026.06 | 83.33 | 230.79 | |
| CoT-SCLLM Backbone=Gemini-2.5-Pro2026.06 | 83.13 | 87.39 | |
| DyLANLLM Backbone=GPT-52026.06 | 82.33 | 37.6 | |
| CoTLLM Backbone=Gemini-2.5-Pro2026.06 | 77.11 | 18.09 | |
| DyLANLLM Backbone=GPT-OSS-120B2026.06 | 75.7 | 0.9 | |
| Claude Opus 4.6Tuple=1, Combos=92026.04 | 74.75 | 4.71 | |
| MAS-ZeroLLM Backbone=GPT-OSS-120B2026.06 | 74.7 | 13.92 | |
| MaASLLM Backbone=GPT-OSS-120B2026.06 | 71.88 | 0.69 | |
| MAS-OrchestraLLM Backbone=GPT-OSS-120B2026.06 | 71.88 | 0.7 | |
| CoT-SCLLM Backbone=GPT-OSS-120B2026.06 | 71.48 | 1.9 | |
| AFlowLLM Backbone=GPT-OSS-120B2026.06 | 71.48 | 10.4 | |
| ADASLLM Backbone=GPT-OSS-120B2026.06 | 71.48 | 3.5 | |
| CoTLLM Backbone=GPT-OSS-120B2026.06 | 70.48 | 0.19 | |
| CoT-SCLLM Backbone=GPT-4o2026.06 | 54.61 | 12.6 | |
| MAS-OrchestraLLM Backbone=GPT-4o2026.06 | 54.22 | 9.3 | |
| CoTLLM Backbone=GPT-4o2026.06 | 53.41 | 1.4 | |
| DyLANLLM Backbone=GPT-4o2026.06 | 53.01 | 7.9 | |
| AFlowLLM Backbone=GPT-4o2026.06 | 52.01 | 130.9 | |
| MaASLLM Backbone=GPT-4o2026.06 | 51.4 | 10.9 | |
| ADASLLM Backbone=GPT-4o2026.06 | 44.4 | 6 | |
| MAS-ZeroLLM Backbone=GPT-4o2026.06 | 43.37 | 259.8 | |
| ReLIFTMethod Category=Comparable Baseline Methods2025.10 | 40.9 | — | |
| LUFFYMethod Category=Comparable Baseline Methods2025.10 | 39.9 | — | |
| Prefix-RFTMethod Category=Comparable Baseline Methods2025.10 | 39.1 | — | |
| SFT+RLMethod Category=SFT and RL2025.10 | 37.9 | — | |
| ICPOMethod Category=Comparable Baseline Methods2025.10 | 34.3 | — | |
| ICPO†Method Category=Comparable Baseline Methods2025.10 | 33.8 | — | |
| RLMethod Category=SFT and RL2025.10 | 32.8 | — | |
| OpenReasoner-Zero*Method Category=Previous RLVR Methods, Result Source=Reported from original papers2025.10 | 29.8 | — | |
| SFTMethod Category=SFT and RL2025.10 | 28.3 | — | |
| Oat-Zero*Method Category=Previous RLVR Methods, Result Source=Reported from original papers2025.10 | 23.7 | — | |
| SimpleRL-Zero*Method Category=Previous RLVR Methods, Result Source=Reported from original papers2025.10 | 23.2 | — | |
| PRIME-Zero*Method Category=Previous RLVR Methods, Result Source=Reported from original papers2025.10 | 18.2 | — | |
| Qwen2.5-Math-7BMethod Category=Base Model2025.10 | 11.1 | — |