Proactive Agent Task on Telepathy-Gym
80.49Pass@U-1Gemini-3-flash
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Gemini-3-flashModel Source=Commercial2026.02 | 80.49 | 82.93 | 85.37 | 14.14 | |
| GPT-5-0807Model Source=Commercial2026.02 | 73.17 | 82.93 | 82.93 | 12.7 | |
| Gemini-2.5-ProModel Source=Commercial2026.02 | 58.54 | 65.85 | 73.17 | 23.3 | |
| Gemini-2.5-FlashModel Source=Commercial2026.02 | 58.53 | 65.85 | 70.73 | 16.25 | |
| BAO-1.7BModel Source=Fine-tuned, Scale=1.7B, Behavior Enhancement=true, Reward Regularization=true2026.02 | 56.1 | 58.54 | 60.97 | 16.01 | |
| Qwen3-32BModel Source=Raw, Scale=32B2026.02 | 53.66 | 56.1 | 58.54 | 19.13 | |
| BAO-4BModel Source=Fine-tuned, Scale=4B, Behavior Enhancement=true, Reward Regularization=true2026.02 | 51.23 | 65.85 | 65.85 | 18.7 | |
| UserRL-4BModel Source=Fine-tuned, Scale=4B2026.02 | 48.78 | 51.22 | 65.85 | 42.51 | |
| BAO w/o BE-4BModel Source=Fine-tuned, Scale=4B, Behavior Enhancement=false, Reward Regularization=true2026.02 | 48.78 | 53.66 | 56.1 | 14.52 | |
| GPT-4oModel Source=Commercial2026.02 | 46.34 | 58.54 | 63.42 | 17.47 | |
| GPT-4o-miniModel Source=Commercial2026.02 | 46.34 | 56.1 | 62.16 | 11.83 | |
| BAO w/o BE-1.7BModel Source=Fine-tuned, Scale=1.7B, Behavior Enhancement=false, Reward Regularization=true2026.02 | 43.9 | 48.78 | 48.78 | 15.59 | |
| Qwen3-14BModel Source=Raw, Scale=14B2026.02 | 39.02 | 46.34 | 46.34 | 25.32 | |
| Qwen3-8BModel Source=Raw, Scale=8B2026.02 | 39.02 | 43.9 | 48.78 | 16.99 | |
| BAO w/o BR-4BModel Source=Fine-tuned, Scale=4B, Behavior Enhancement=true, Reward Regularization=false2026.02 | 39.02 | 43.9 | 56.1 | 52.9 | |
| UserRL-1.7BModel Source=Fine-tuned, Scale=1.7B2026.02 | 26.83 | 26.83 | 43.9 | 65.43 | |
| BAO w/o BR-1.7BModel Source=Fine-tuned, Scale=1.7B, Behavior Enhancement=true, Reward Regularization=false2026.02 | 17.07 | 19.51 | 43.9 | 69.11 |