Task success rate on DMV-Bench J=5 1.0
82.7Task Success RateDualMem
Evaluation results
| Method | Links | |
|---|---|---|
| DualMemBack-end=Gemini 2.5 Flash, Steps (stp)=118–134, Probe counts (nr)=2,7622026.06 | 82.7 | |
| DualMemBack-end=Qwen2.5-VL-7B, Steps (stp)=118–134, Probe counts (nr)=1,0532026.06 | 81.1 | |
| M2ABack-end=Qwen2.5-VL-7B, Steps (stp)=118–134, Probe counts (nr)=1,0532026.06 | 70.4 | |
| CaptionBack-end=Qwen2.5-VL-7B, Steps (stp)=118–134, Probe counts (nr)=1,0532026.06 | 67.3 | |
| M2ABack-end=Gemini 2.5 Flash, Steps (stp)=118–134, Probe counts (nr)=2,7622026.06 | 65.7 | |
| CaptionBack-end=Gemini 2.5 Flash, Steps (stp)=118–134, Probe counts (nr)=2,7622026.06 | 58.9 | |
| MMABack-end=Qwen2.5-VL-7B, Steps (stp)=118–134, Probe counts (nr)=1,0532026.06 | 47.7 | |
| MMABack-end=Gemini 2.5 Flash, Steps (stp)=118–134, Probe counts (nr)=2,7622026.06 | 46.1 | |
| WorldMMBack-end=Gemini 2.5 Flash, Steps (stp)=118–134, Probe counts (nr)=2,7622026.06 | 43.5 | |
| WorldMMBack-end=Qwen2.5-VL-7B, Steps (stp)=118–134, Probe counts (nr)=1,0532026.06 | 39.8 | |
| NoMemoryBack-end=Qwen2.5-VL-7B, Steps (stp)=118–134, Probe counts (nr)=1,0532026.06 | 0 | |
| TextOnlyBack-end=Qwen2.5-VL-7B, Steps (stp)=118–134, Probe counts (nr)=1,0532026.06 | 0 | |
| NoMemoryBack-end=Gemini 2.5 Flash, Steps (stp)=118–134, Probe counts (nr)=2,7622026.06 | 0 | |
| TextOnlyBack-end=Gemini 2.5 Flash, Steps (stp)=118–134, Probe counts (nr)=2,7622026.06 | 0 |