Interactive Task Success on Spider2 Snow
69.29Success RateMERIT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MERITBackbone=GPT-5, User Simulator=GPT-4o, Memory Type=Dynamic Memory2026.05 | 69.29 | 13.44 | |
| MementoBackbone=GPT-5, User Simulator=GPT-4o, Memory Type=Dynamic Memory2026.05 | 68.56 | 14.33 | |
| Cosine SimilarityBackbone=GPT-5, User Simulator=GPT-4o, Memory Type=Static Memory2026.05 | 68.19 | 13.48 | |
| BM25Backbone=GPT-5, User Simulator=GPT-4o, Memory Type=Static Memory2026.05 | 67.09 | 14.58 | |
| MemRLBackbone=GPT-5, User Simulator=GPT-4o, Memory Type=Dynamic Memory2026.05 | 66.91 | 14.95 | |
| TopicKBackbone=GPT-5, User Simulator=GPT-4o, Memory Type=Static Memory2026.05 | 66.73 | 14.54 | |
| No MemoryBackbone=GPT-5, User Simulator=GPT-4o, Memory Type=No Memory2026.05 | 66 | 13.29 | |
| Vanilla GPT-5Backbone=GPT-5, User Simulator=GPT-4o, Memory Type=No Memory2026.05 | 37.29 | — |