Proactive Agent Task on Turtle-Gym
35Pass@U-1GPT-5-0807
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-5-0807Model Source=Commercial2026.02 | 35 | 35.21 | 0.3521 | 8.97 | |
| Gemini-2.5-ProModel Source=Commercial2026.02 | 31.56 | 34.69 | 0.3844 | 22.5 | |
| Gemini-3-flashModel Source=Commercial2026.02 | 18.23 | 20.73 | 0.2073 | 6.13 | |
| GPT-4oModel Source=Commercial2026.02 | 17.19 | 23.33 | 0.275 | 25.53 | |
| Gemini-2.5-FlashModel Source=Commercial2026.02 | 16.04 | 18.44 | 0.2198 | 21.55 | |
| Qwen3-14BModel Source=Raw, Scale=14B2026.02 | 12.81 | 14.06 | 0.15 | 52.64 | |
| BAO-4BModel Source=Fine-tuned, Scale=4B, Behavior Enhancement=true, Reward Regularization=true2026.02 | 10.63 | 11.25 | 0.1125 | 29.17 | |
| Qwen3-32BModel Source=Raw, Scale=32B2026.02 | 10 | 15 | 0.1688 | 30.82 | |
| BAO w/o BE-4BModel Source=Fine-tuned, Scale=4B, Behavior Enhancement=false, Reward Regularization=true2026.02 | 8.02 | 9.79 | 0.1146 | 33.59 | |
| Qwen3-8BModel Source=Raw, Scale=8B2026.02 | 7.92 | 9.38 | 0.1135 | 47.78 | |
| BAO w/o BR-4BModel Source=Fine-tuned, Scale=4B, Behavior Enhancement=true, Reward Regularization=false2026.02 | 7.29 | 7.5 | 0.0813 | 86.98 | |
| BAO w/o BE-1.7BModel Source=Fine-tuned, Scale=1.7B, Behavior Enhancement=false, Reward Regularization=true2026.02 | 6.15 | 6.6 | 0.075 | 37.76 | |
| GPT-4o-miniModel Source=Commercial2026.02 | 5.98 | 7.32 | 0.0793 | 33.23 | |
| BAO w/o BR-1.7BModel Source=Fine-tuned, Scale=1.7B, Behavior Enhancement=true, Reward Regularization=false2026.02 | 5.94 | 6.25 | 0.0771 | 90.64 | |
| BAO-1.7BModel Source=Fine-tuned, Scale=1.7B, Behavior Enhancement=true, Reward Regularization=true2026.02 | 5.63 | 6.67 | 0.0667 | 29.76 | |
| UserRL-1.7BModel Source=Fine-tuned, Scale=1.7B2026.02 | 4.79 | 5.1 | 0.0583 | 77.21 | |
| UserRL-4BModel Source=Fine-tuned, Scale=4B2026.02 | 4.17 | 5.63 | 0.0594 | 76.17 |