# Task success rate on DMV-Bench J=15 1.0 Source: [Task success rate on DMV-Bench J=15 1.0]() Collection index: [llms.txt]() Data updated: 2026-06-29T06:30:42.322Z Evaluation of agent performance on the DMV-Bench J=15 1.0 dataset, measured by the Task Success Rate. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Task success rate | | Dataset (including any stated split) | DMV-Bench J=15 1.0 | | Metric | Task Success Rate | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 14 of 14 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Task Success Rate](): Higher is better; unit: PERCENT. ## Recorded results | Method | Task Success Rate | Result date | Report paper | | --- | --- | --- | --- | | DualMem | 75.1 | Not recorded | 6a41fc15206542251d7ccb18 | | DualMem | 71.3 | Not recorded | 6a41fc15206542251d7ccb18 | | M2A | 62.6 | Not recorded | 6a41fc15206542251d7ccb18 | | Caption | 62.3 | Not recorded | 6a41fc15206542251d7ccb18 | | M2A | 59.6 | Not recorded | 6a41fc15206542251d7ccb18 | | Caption | 50.7 | Not recorded | 6a41fc15206542251d7ccb18 | | MMA | 39.4 | Not recorded | 6a41fc15206542251d7ccb18 | | WorldMM | 38.4 | Not recorded | 6a41fc15206542251d7ccb18 | | MMA | 33.6 | Not recorded | 6a41fc15206542251d7ccb18 | | WorldMM | 29.7 | Not recorded | 6a41fc15206542251d7ccb18 | | NoMemory | 0 | Not recorded | 6a41fc15206542251d7ccb18 | | TextOnly | 0 | Not recorded | 6a41fc15206542251d7ccb18 | | NoMemory | 0 | Not recorded | 6a41fc15206542251d7ccb18 | | TextOnly | 0 | Not recorded | 6a41fc15206542251d7ccb18 | ### DualMem Task Success Rate: 75.1. Result ID: 6a42110afbc373592c8662c7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Back-end":"Qwen2.5-VL-7B","Steps (stp)":"359–394","Probe counts (nr)":"10,307"}. This record does not assert independent reproduction. ### DualMem Task Success Rate: 71.3. Result ID: 6a42110afbc373592c8662ce. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Back-end":"Gemini 2.5 Flash","Steps (stp)":"359–394","Probe counts (nr)":"28,710"}. This record does not assert independent reproduction. ### M2A Task Success Rate: 62.6. Result ID: 6a42110afbc373592c8662c6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Back-end":"Qwen2.5-VL-7B","Steps (stp)":"359–394","Probe counts (nr)":"10,307"}. This record does not assert independent reproduction. ### Caption Task Success Rate: 62.3. Result ID: 6a42110afbc373592c8662c3. [Reporting paper]() Recorded evaluation configuration: {"Back-end":"Qwen2.5-VL-7B","Steps (stp)":"359–394","Probe counts (nr)":"10,307"}. This record does not assert independent reproduction. ### M2A Task Success Rate: 59.6. Result ID: 6a42110afbc373592c8662cd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Back-end":"Gemini 2.5 Flash","Steps (stp)":"359–394","Probe counts (nr)":"28,710"}. This record does not assert independent reproduction. ### Caption Task Success Rate: 50.7. Result ID: 6a42110afbc373592c8662ca. [Reporting paper]() Recorded evaluation configuration: {"Back-end":"Gemini 2.5 Flash","Steps (stp)":"359–394","Probe counts (nr)":"28,710"}. This record does not assert independent reproduction. ### MMA Task Success Rate: 39.4. Result ID: 6a42110afbc373592c8662c5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Back-end":"Qwen2.5-VL-7B","Steps (stp)":"359–394","Probe counts (nr)":"10,307"}. This record does not assert independent reproduction. ### WorldMM Task Success Rate: 38.4. Result ID: 6a42110afbc373592c8662cb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Back-end":"Gemini 2.5 Flash","Steps (stp)":"359–394","Probe counts (nr)":"28,710"}. This record does not assert independent reproduction. ### MMA Task Success Rate: 33.6. Result ID: 6a42110afbc373592c8662cc. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Back-end":"Gemini 2.5 Flash","Steps (stp)":"359–394","Probe counts (nr)":"28,710"}. This record does not assert independent reproduction. ### WorldMM Task Success Rate: 29.7. Result ID: 6a42110afbc373592c8662c4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Back-end":"Qwen2.5-VL-7B","Steps (stp)":"359–394","Probe counts (nr)":"10,307"}. This record does not assert independent reproduction. ### NoMemory Task Success Rate: 0. Result ID: 6a42110afbc373592c8662c1. [Reporting paper]() Recorded evaluation configuration: {"Back-end":"Qwen2.5-VL-7B","Steps (stp)":"359–394","Probe counts (nr)":"10,307"}. This record does not assert independent reproduction. ### TextOnly Task Success Rate: 0. Result ID: 6a42110afbc373592c8662c2. [Reporting paper]() Recorded evaluation configuration: {"Back-end":"Qwen2.5-VL-7B","Steps (stp)":"359–394","Probe counts (nr)":"10,307"}. This record does not assert independent reproduction. ### NoMemory Task Success Rate: 0. Result ID: 6a42110afbc373592c8662c8. [Reporting paper]() Recorded evaluation configuration: {"Back-end":"Gemini 2.5 Flash","Steps (stp)":"359–394","Probe counts (nr)":"28,710"}. This record does not assert independent reproduction. ### TextOnly Task Success Rate: 0. Result ID: 6a42110afbc373592c8662c9. [Reporting paper]() Recorded evaluation configuration: {"Back-end":"Gemini 2.5 Flash","Steps (stp)":"359–394","Probe counts (nr)":"28,710"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.