# Task success rate on DMV-Bench MC J=50 1.0 Source: [Task success rate on DMV-Bench MC J=50 1.0]() Collection index: [llms.txt]() Data updated: 2026-06-29T06:30:42.349Z Evaluation of agent performance on the DMV-Bench MC J=50 1.0 dataset, measured by Task Success Rate. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Task success rate | | Dataset (including any stated split) | DMV-Bench MC J=50 1.0 | | Metric | Task Success Rate | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 14 of 14 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Task Success Rate](): Higher is better; unit: PERCENT. ## Recorded results | Method | Task Success Rate | Result date | Report paper | | --- | --- | --- | --- | | DualMem | 68.3 | Not recorded | 6a41fc15206542251d7ccb18 | | DualMem | 65.1 | Not recorded | 6a41fc15206542251d7ccb18 | | M2A | 64.7 | Not recorded | 6a41fc15206542251d7ccb18 | | Caption | 58.8 | Not recorded | 6a41fc15206542251d7ccb18 | | M2A | 58.7 | Not recorded | 6a41fc15206542251d7ccb18 | | Caption | 47.7 | Not recorded | 6a41fc15206542251d7ccb18 | | WorldMM | 37 | Not recorded | 6a41fc15206542251d7ccb18 | | MMA | 36.9 | Not recorded | 6a41fc15206542251d7ccb18 | | MMA | 35.6 | Not recorded | 6a41fc15206542251d7ccb18 | | WorldMM | 30.9 | Not recorded | 6a41fc15206542251d7ccb18 | | NoMemory | 0 | Not recorded | 6a41fc15206542251d7ccb18 | | TextOnly | 0 | Not recorded | 6a41fc15206542251d7ccb18 | | NoMemory | 0 | Not recorded | 6a41fc15206542251d7ccb18 | | TextOnly | 0 | Not recorded | 6a41fc15206542251d7ccb18 | ### DualMem Task Success Rate: 68.3. Result ID: 6a42110afbc373592c8662d5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Back-end":"Qwen2.5-VL-7B","Monte Carlo range (r)":"1–49","Probe counts (nr)":"2,407"}. This record does not assert independent reproduction. ### DualMem Task Success Rate: 65.1. Result ID: 6a42110afbc373592c8662dc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Back-end":"Gemini 2.5 Flash","Monte Carlo range (r)":"1–49","Probe counts (nr)":"2,449"}. This record does not assert independent reproduction. ### M2A Task Success Rate: 64.7. Result ID: 6a42110afbc373592c8662db. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Back-end":"Gemini 2.5 Flash","Monte Carlo range (r)":"1–49","Probe counts (nr)":"2,449"}. This record does not assert independent reproduction. ### Caption Task Success Rate: 58.8. Result ID: 6a42110afbc373592c8662d1. [Reporting paper]() Recorded evaluation configuration: {"Back-end":"Qwen2.5-VL-7B","Monte Carlo range (r)":"1–49","Probe counts (nr)":"2,407"}. This record does not assert independent reproduction. ### M2A Task Success Rate: 58.7. Result ID: 6a42110afbc373592c8662d4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Back-end":"Qwen2.5-VL-7B","Monte Carlo range (r)":"1–49","Probe counts (nr)":"2,407"}. This record does not assert independent reproduction. ### Caption Task Success Rate: 47.7. Result ID: 6a42110afbc373592c8662d8. [Reporting paper]() Recorded evaluation configuration: {"Back-end":"Gemini 2.5 Flash","Monte Carlo range (r)":"1–49","Probe counts (nr)":"2,449"}. This record does not assert independent reproduction. ### WorldMM Task Success Rate: 37. Result ID: 6a42110afbc373592c8662d9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Back-end":"Gemini 2.5 Flash","Monte Carlo range (r)":"1–49","Probe counts (nr)":"2,449"}. This record does not assert independent reproduction. ### MMA Task Success Rate: 36.9. Result ID: 6a42110afbc373592c8662da. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Back-end":"Gemini 2.5 Flash","Monte Carlo range (r)":"1–49","Probe counts (nr)":"2,449"}. This record does not assert independent reproduction. ### MMA Task Success Rate: 35.6. Result ID: 6a42110afbc373592c8662d3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Back-end":"Qwen2.5-VL-7B","Monte Carlo range (r)":"1–49","Probe counts (nr)":"2,407"}. This record does not assert independent reproduction. ### WorldMM Task Success Rate: 30.9. Result ID: 6a42110afbc373592c8662d2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Back-end":"Qwen2.5-VL-7B","Monte Carlo range (r)":"1–49","Probe counts (nr)":"2,407"}. This record does not assert independent reproduction. ### NoMemory Task Success Rate: 0. Result ID: 6a42110afbc373592c8662cf. [Reporting paper]() Recorded evaluation configuration: {"Back-end":"Qwen2.5-VL-7B","Monte Carlo range (r)":"1–49","Probe counts (nr)":"2,407"}. This record does not assert independent reproduction. ### TextOnly Task Success Rate: 0. Result ID: 6a42110afbc373592c8662d0. [Reporting paper]() Recorded evaluation configuration: {"Back-end":"Qwen2.5-VL-7B","Monte Carlo range (r)":"1–49","Probe counts (nr)":"2,407"}. This record does not assert independent reproduction. ### NoMemory Task Success Rate: 0. Result ID: 6a42110afbc373592c8662d6. [Reporting paper]() Recorded evaluation configuration: {"Back-end":"Gemini 2.5 Flash","Monte Carlo range (r)":"1–49","Probe counts (nr)":"2,449"}. This record does not assert independent reproduction. ### TextOnly Task Success Rate: 0. Result ID: 6a42110afbc373592c8662d7. [Reporting paper]() Recorded evaluation configuration: {"Back-end":"Gemini 2.5 Flash","Monte Carlo range (r)":"1–49","Probe counts (nr)":"2,449"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.