Strategy Prediction on ESC (test)
0.67Error Rate Metric (EMR)GPT-5
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| GPT-5Model Group=Instruction-following LLMs2026.04 | 0.67 | 12.26 | 32.04 | |
| DeepSeek-R1Model Group=Instruction-following LLMs2026.04 | 12.85 | 18.61 | 14.07 | |
| Qwen3-235b-InstructModel Group=Instruction-following LLMs2026.04 | 15.15 | 19.77 | 13.69 | |
| All-In-OneBase Model=LLaMA3.1-8b2026.04 | 23.61 | 28.63 | 12.5 | |
| One-by-OneBase Model=LLaMA3.1-8b2026.04 | 24.99 | 30.15 | 12.66 | |
| Single StrategyBase Model=LLaMA3.1-8b2026.04 | 25.21 | 28.28 | 12.7 | |
| One-by-One + ReasoningBase Model=LLaMA3.1-8b, reasoning=true2026.04 | 29.55 | 34.3 | 12.34 | |
| All-In-One + ReasoningBase Model=LLaMA3.1-8b, reasoning=true2026.04 | 29.72 | 34.92 | 12.37 | |
| All-In-One + Reasoning + RLBase Model=LLaMA3.1-8b, reasoning=true, RL=true2026.04 | 29.97 | 36.22 | 12.43 | |
| One-by-One + Reasoning + RLBase Model=LLaMA3.1-8b, reasoning=true, RL=true2026.04 | 33.53 | 37.97 | 12.13 |