Reasoning on superGPQA (Accuracy)
39.3Accuracy (superGPQA)TextGrad
Evaluation Results
| Method | Links | |
|---|---|---|
| TextGradActor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_TG → πA2026.04 | 39.3 | |
| DSPyActor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_DSPy → πA2026.04 | 39.3 | |
| Inference-time scaling (ITS)Actor Model=gpt-oss-120b, Supplement Model=None, Supplement Generation Method=∅ → πA_ITS2026.04 | 38.5 | |
| SGT (DPO Iteration 3)Actor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_3 → πA2026.04 | 38.5 | |
| SGT (DPO Iteration 4)Actor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_4 → πA2026.04 | 38.5 | |
| SGT (DPO Iteration 5)Actor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_5 → πA2026.04 | 38.5 | |
| Baseline ActorActor Model=gpt-oss-120b, Supplement Model=None, Supplement Generation Method=∅ → πA2026.04 | 37.2 | |
| SFT SupplementActor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_SFT → πA2026.04 | 36.5 | |
| SGT (DPO Iteration 1)Actor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_1 → πA2026.04 | 33.8 | |
| SGT (DPO Iteration 2)Actor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_2 → πA2026.04 | 33.8 | |
| Inference-time scaling (ITS)Actor Model=v3.5-sonnet-v2, Supplement Model=None, Supplement Generation Method=∅ → πA_ITS2026.04 | 29.5 | |
| Prompted SupplementActor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_prompt → πA2026.04 | 28 | |
| SGT (DPO Iteration 5)Actor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_5 → πA2026.04 | 24.2 | |
| SGT (DPO Iteration 4)Actor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_4 → πA2026.04 | 21.2 | |
| SGT (DPO Iteration 2)Actor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_2 → πA2026.04 | 20.5 | |
| SGT (DPO Iteration 3)Actor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_3 → πA2026.04 | 20.5 | |
| Baseline ActorActor Model=v3.5-sonnet-v2, Supplement Model=None, Supplement Generation Method=∅ → πA2026.04 | 20.3 | |
| TextGradActor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_TG → πA2026.04 | 20.3 | |
| DSPyActor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_DSPy → πA2026.04 | 20 | |
| SGT (DPO Iteration 1)Actor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_1 → πA2026.04 | 19.8 | |
| Prompted SupplementActor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_prompt → πA2026.04 | 18 | |
| SFT SupplementActor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=πS_SFT → πA2026.04 | 18 | |
| Supplement Solver OnlyActor Model=v3.5-sonnet-v2, Supplement Model=Qwen3-1.7B, Supplement Generation Method=∅ → πS_solve2026.04 | 9.2 | |
| Supplement Solver OnlyActor Model=gpt-oss-120b, Supplement Model=Qwen3-1.7B, Supplement Generation Method=∅ → πS_solve2026.04 | 9.2 |