Interactive Database Interaction on BIRD Interact
7.09Avg Turns (WRITE)MERIT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MERITMemory Strategy=Dynamic Memory, Backbone=GPT-5, User Simulator=GPT-4o2026.05 | 7.09 | 8.15 | 7.84 | |
| TopicKMemory Strategy=Static Memory, Backbone=GPT-5, User Simulator=GPT-4o2026.05 | 7.46 | 8.64 | 8.3 | |
| MemRLMemory Strategy=Dynamic Memory, Backbone=GPT-5, User Simulator=GPT-4o2026.05 | 7.58 | 8.79 | 8.44 | |
| Cosine SimilarityMemory Strategy=Static Memory, Backbone=GPT-5, User Simulator=GPT-4o2026.05 | 7.63 | 8.79 | 8.45 | |
| BM25Memory Strategy=Static Memory, Backbone=GPT-5, User Simulator=GPT-4o2026.05 | 7.65 | 8.53 | 8.27 | |
| MementoMemory Strategy=Dynamic Memory, Backbone=GPT-5, User Simulator=GPT-4o2026.05 | 7.8 | 8.91 | 8.59 | |
| No MemoryMemory Strategy=No Memory, Backbone=GPT-5, User Simulator=GPT-4o2026.05 | 8.24 | 9.21 | 8.93 | |
| DemoMemory Strategy=No Memory, Backbone=GPT-5, User Simulator=GPT-4o2026.05 | 8.44 | 9.93 | 9.5 |