Task success rate on DMV-Bench J=15 1.0
75.1Task Success RateDualMem
Evaluation results
| Method | Links | |
|---|---|---|
| DualMemBack-end=Qwen2.5-VL-7B, Steps (stp)=359–394, Probe counts (nr)=10,3072026.06 | 75.1 | |
| DualMemBack-end=Gemini 2.5 Flash, Steps (stp)=359–394, Probe counts (nr)=28,7102026.06 | 71.3 | |
| M2ABack-end=Qwen2.5-VL-7B, Steps (stp)=359–394, Probe counts (nr)=10,3072026.06 | 62.6 | |
| CaptionBack-end=Qwen2.5-VL-7B, Steps (stp)=359–394, Probe counts (nr)=10,3072026.06 | 62.3 | |
| M2ABack-end=Gemini 2.5 Flash, Steps (stp)=359–394, Probe counts (nr)=28,7102026.06 | 59.6 | |
| CaptionBack-end=Gemini 2.5 Flash, Steps (stp)=359–394, Probe counts (nr)=28,7102026.06 | 50.7 | |
| MMABack-end=Qwen2.5-VL-7B, Steps (stp)=359–394, Probe counts (nr)=10,3072026.06 | 39.4 | |
| WorldMMBack-end=Gemini 2.5 Flash, Steps (stp)=359–394, Probe counts (nr)=28,7102026.06 | 38.4 | |
| MMABack-end=Gemini 2.5 Flash, Steps (stp)=359–394, Probe counts (nr)=28,7102026.06 | 33.6 | |
| WorldMMBack-end=Qwen2.5-VL-7B, Steps (stp)=359–394, Probe counts (nr)=10,3072026.06 | 29.7 | |
| NoMemoryBack-end=Qwen2.5-VL-7B, Steps (stp)=359–394, Probe counts (nr)=10,3072026.06 | 0 | |
| TextOnlyBack-end=Qwen2.5-VL-7B, Steps (stp)=359–394, Probe counts (nr)=10,3072026.06 | 0 | |
| NoMemoryBack-end=Gemini 2.5 Flash, Steps (stp)=359–394, Probe counts (nr)=28,7102026.06 | 0 | |
| TextOnlyBack-end=Gemini 2.5 Flash, Steps (stp)=359–394, Probe counts (nr)=28,7102026.06 | 0 |