Task success rate on DMV-Bench J=10 1.0
77.2Task Success RateDualMem
Evaluation results
| Method | Links | |
|---|---|---|
| DualMemBack-end=Qwen2.5-VL-7B, Steps (stp)=238–259, Probe counts (nr)=4,8212026.06 | 77.2 | |
| DualMemBack-end=Gemini 2.5 Flash, Steps (stp)=238–259, Probe counts (nr)=12,3442026.06 | 75.2 | |
| M2ABack-end=Qwen2.5-VL-7B, Steps (stp)=238–259, Probe counts (nr)=4,8212026.06 | 64.8 | |
| CaptionBack-end=Qwen2.5-VL-7B, Steps (stp)=238–259, Probe counts (nr)=4,8212026.06 | 63.7 | |
| M2ABack-end=Gemini 2.5 Flash, Steps (stp)=238–259, Probe counts (nr)=12,3442026.06 | 63 | |
| CaptionBack-end=Gemini 2.5 Flash, Steps (stp)=238–259, Probe counts (nr)=12,3442026.06 | 53.4 | |
| MMABack-end=Gemini 2.5 Flash, Steps (stp)=238–259, Probe counts (nr)=12,3442026.06 | 41.7 | |
| MMABack-end=Qwen2.5-VL-7B, Steps (stp)=238–259, Probe counts (nr)=4,8212026.06 | 41 | |
| WorldMMBack-end=Gemini 2.5 Flash, Steps (stp)=238–259, Probe counts (nr)=12,3442026.06 | 39.3 | |
| WorldMMBack-end=Qwen2.5-VL-7B, Steps (stp)=238–259, Probe counts (nr)=4,8212026.06 | 32.6 | |
| NoMemoryBack-end=Qwen2.5-VL-7B, Steps (stp)=238–259, Probe counts (nr)=4,8212026.06 | 0 | |
| TextOnlyBack-end=Qwen2.5-VL-7B, Steps (stp)=238–259, Probe counts (nr)=4,8212026.06 | 0 | |
| NoMemoryBack-end=Gemini 2.5 Flash, Steps (stp)=238–259, Probe counts (nr)=12,3442026.06 | 0 | |
| TextOnlyBack-end=Gemini 2.5 Flash, Steps (stp)=238–259, Probe counts (nr)=12,3442026.06 | 0 |