Retrieval-Augmented Question Answering on DeepSearch Musique (SR, Tokens (K))
46SRSynapse
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SynapseTaskExecutor=gpt-4o2026.04 | 46 | 38.9 | |
| SynapseTaskExecutor=Gemini-3.0-flash2026.04 | 44 | 83 | |
| SynapseTaskExecutor=gpt-4o-mini, Category=Baselines & Retrieval-Based Memory2026.04 | 42 | 41.8 | |
| SynapseTaskExecutor=Gemini-2.5-flash2026.04 | 40 | 77.8 | |
| ActiveContextTaskExecutor=Gemini-3.0-flash2026.04 | 36 | 8.5 | |
| Full ContextTaskExecutor=Gemini-3.0-flash2026.04 | 33 | 83.6 | |
| Training-Free GRPOTaskExecutor=Gemini-3.0-flash2026.04 | 27 | 82.7 | |
| ActiveContextTaskExecutor=Gemini-2.5-flash2026.04 | 25.6 | 8.3 | |
| Gemini-3.0-flashTaskExecutor=gpt-4o-mini, Category=Zero-Shot Curators2026.04 | 19 | 9.6 | |
| AWMTaskExecutor=gpt-4o-mini, Category=Baselines & Retrieval-Based Memory2026.04 | 18 | 43.2 | |
| Training-Free GRPOTaskExecutor=Gemini-2.5-flash2026.04 | 17 | 119 | |
| Gemini-2.5-flashTaskExecutor=gpt-4o-mini, Category=Zero-Shot Curators2026.04 | 17 | 9.7 | |
| gpt-4oTaskExecutor=gpt-4o-mini, Category=Zero-Shot Curators2026.04 | 17 | 13.4 | |
| gpt-4o-miniTaskExecutor=gpt-4o-mini, Category=Zero-Shot Curators2026.04 | 17 | 12.6 | |
| Full ContextTaskExecutor=Gemini-2.5-flash2026.04 | 16 | 42 | |
| Full ContextTaskExecutor=gpt-4o2026.04 | 16 | 42 | |
| Training-Free GRPOTaskExecutor=gpt-4o-mini, Category=Baselines & Retrieval-Based Memory2026.04 | 16 | 45.4 | |
| Training-Free GRPOTaskExecutor=gpt-4o2026.04 | 15 | 42.8 | |
| ActiveContextTaskExecutor=gpt-4o2026.04 | 15 | 9.1 | |
| Full ContextTaskExecutor=gpt-4o-mini, Category=Baselines & Retrieval-Based Memory2026.04 | 15 | 42.1 | |
| Mem0TaskExecutor=gpt-4o-mini, Category=Baselines & Retrieval-Based Memory2026.04 | 15 | 59.7 | |
| ContextCuratorTaskExecutor=gpt-4o-mini, Category=RL-Trained Curator2026.04 | 12 | 9.3 | |
| Qwen2.5-7B-InstructTaskExecutor=gpt-4o-mini, Category=Zero-Shot Curators2026.04 | 10 | 12.9 |