Multi-turn tool-use on AppWorld
63.6Avg@4SeeUPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SeeUPOBackbone=Qwen3-14B, Fine-tuning=RLHF2026.02 | 63.6 | 80.7 | |
| SeeUPOBackbone=Qwen2.5-14B, Fine-tuning=RLHF2026.02 | 50.88 | 70.18 | |
| ReMe (dynamic)Model=Qwen3-32B2025.12 | 42.02 | 63.49 | |
| PPOBackbone=Qwen2.5-14B, Fine-tuning=RLHF2026.02 | 40.79 | 57.89 | |
| GRPOBackbone=Qwen3-14B, Fine-tuning=RLHF2026.02 | 40.35 | 57.89 | |
| GRPOBackbone=Qwen2.5-14B, Fine-tuning=RLHF2026.02 | 35.53 | 49.12 | |
| PPOBackbone=Qwen3-14B, Fine-tuning=RLHF2026.02 | 35.09 | 54.39 | |
| ReMe (dynamic)Model=Qwen3-14B2025.12 | 34.32 | 52.98 | |
| GSPOBackbone=Qwen3-14B, Fine-tuning=RLHF2026.02 | 32.89 | 52.63 | |
| ReMe (fixed)Model=Qwen3-32B2025.12 | 31.5 | 58.13 | |
| A-MemModel=Qwen3-32B2025.12 | 28.13 | 51.19 | |
| No MemoryModel=Qwen3-32B2025.12 | 27.23 | 50.59 | |
| ReMe (fixed)Model=Qwen3-14B2025.12 | 25.35 | 46.82 | |
| ReMe (dynamic)Model=Qwen3-8B2025.12 | 24.7 | 42.06 | |
| LangMemModel=Qwen3-32B2025.12 | 24.55 | 47.02 | |
| GSPOBackbone=Qwen2.5-14B, Fine-tuning=RLHF2026.02 | 24.12 | 38.6 | |
| No MemoryModel=Qwen3-14B2025.12 | 22.57 | 41.07 | |
| LangMemModel=Qwen3-14B2025.12 | 21.88 | 41.67 | |
| Qwen3-14BBackbone=Qwen3-14B, Fine-tuning=None2026.02 | 20.18 | 35.09 | |
| A-MemModel=Qwen3-14B2025.12 | 18.95 | 37.7 | |
| ReMe (fixed)Model=Qwen3-8B2025.12 | 17.06 | 36.31 | |
| No MemoryModel=Qwen3-8B2025.12 | 14.97 | 32.85 | |
| A-MemModel=Qwen3-8B2025.12 | 12.95 | 29.76 | |
| LangMemModel=Qwen3-8B2025.12 | 11.46 | 26.79 | |
| Qwen2.5-14BBackbone=Qwen2.5-14B, Fine-tuning=None2026.02 | 4.825 | 7.018 |