# Procedural Multimodal Machine Comprehension on CraftQA (test) Source: [Procedural Multimodal Machine Comprehension on CraftQA (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:39:25.223Z Evaluation of Procedural Multimodal Machine Comprehension performance on the test split of the CraftQA dataset, reporting metrics such as Cloze, Coherence, and Ordering scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Procedural Multimodal Machine Comprehension | | Dataset (including any stated split) | CraftQA (test) | | Metric | Cloze | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 10 of 10 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Cloze](): Higher is better; unit: PERCENT. - [Coherence](): Higher is better; unit: PERCENT. - [Ordering](): Higher is better; unit: PERCENT. - [Average Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Cloze | Result date | Report paper | | --- | --- | --- | --- | | Human | 55.33 | Not recorded | 68fa6c2d1e209131cea3290a | | TMEG | 48.01 | Not recorded | 68fa6c2d1e209131cea3290a | | TMEG | 47.49 | Not recorded | 68fa6c2d1e209131cea3290a | | TMEG | 46.67 | Not recorded | 68fa6c2d1e209131cea3290a | | TMEG | 45.91 | Not recorded | 68fa6c2d1e209131cea3290a | | TMEG | 44.87 | Not recorded | 68fa6c2d1e209131cea3290a | | VisualBERT | 40.56 | Not recorded | 68fa6c2d1e209131cea3290a | | MLMM-Trans | 39.43 | Not recorded | 68fa6c2d1e209131cea3290a | | PRN | 35.14 | Not recorded | 68fa6c2d1e209131cea3290a | | HS | 27.81 | Not recorded | 68fa6c2d1e209131cea3290a | ### Human Cloze: 55.33. Result ID: 698ee9883a6c314849f35923. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### TMEG Cloze: 48.01. Result ID: 698ee9883a6c314849f35928. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### TMEG Cloze: 47.49. Result ID: 698ee9883a6c314849f3592c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Contrastive Coherence Loss":false}. This record does not assert independent reproduction. ### TMEG Cloze: 46.67. Result ID: 698ee9883a6c314849f3592a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Modal Encoding":false}. This record does not assert independent reproduction. ### TMEG Cloze: 45.91. Result ID: 698ee9883a6c314849f35929. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Temporal Encoding":false}. This record does not assert independent reproduction. ### TMEG Cloze: 44.87. Result ID: 698ee9883a6c314849f3592b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Temporal Encoding":false,"Modal Encoding":false}. This record does not assert independent reproduction. ### VisualBERT Cloze: 40.56. Result ID: 698ee9883a6c314849f35927. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### MLMM-Trans Cloze: 39.43. Result ID: 698ee9883a6c314849f35926. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### PRN Cloze: 35.14. Result ID: 698ee9883a6c314849f35925. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### HS Cloze: 27.81. Result ID: 698ee9883a6c314849f35924. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.