# Multimodal Understanding and QA on MME, OCRBench, DocVQA, RealWorldQA, and VLMBlind Source: [Multimodal Understanding and QA on MME, OCRBench, DocVQA, RealWorldQA, and VLMBlind]() Collection index: [llms.txt]() Data updated: 2026-03-18T09:01:43.106Z Aggregate evaluation covering performance across five distinct multimodal benchmarks: MME, OCRBench, DocVQA, RealWorldQA, and VLMBlind, often including a recovery metric. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multimodal Understanding and Question Answering | | Dataset (including any stated split) | Multimodal Benchmarks MME, OCRBench, DocVQA, RealWorldQA, VLMBlind | | Metric | MME Score | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 33 of 33 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [MME Score](): Higher is better; unit: SCALAR. - [OCRBench Score](): Higher is better; unit: SCALAR. - [DocVQA Accuracy](): Higher is better; unit: PERCENT. - [RealWorldQA Accuracy](): Higher is better; unit: PERCENT. - [VLMBlind Accuracy](): Higher is better; unit: PERCENT. - [Recovery Rate](): Higher is better; unit: PERCENT. ## Recorded results | Method | MME Score | Result date | Report paper | | --- | --- | --- | --- | | BATQuant | 2386 | Not recorded | 69ba31900636e5bf4afc0884 | | BF16 | 2377 | Not recorded | 69ba31900636e5bf4afc0884 | | BATQuant | 2368 | Not recorded | 69ba31900636e5bf4afc0884 | | BATQuant | 2360 | Not recorded | 69ba31900636e5bf4afc0884 | | FlatQuant | 2353 | Not recorded | 69ba31900636e5bf4afc0884 | | FlatQuant | 2351 | Not recorded | 69ba31900636e5bf4afc0884 | | SmoothQuant | 2349 | Not recorded | 69ba31900636e5bf4afc0884 | | GPTQ | 2346 | Not recorded | 69ba31900636e5bf4afc0884 | | GPTQ | 2340 | Not recorded | 69ba31900636e5bf4afc0884 | | BATQuant | 2332 | Not recorded | 69ba31900636e5bf4afc0884 | | BRQ | 2329 | Not recorded | 69ba31900636e5bf4afc0884 | | GPTQ | 2328 | Not recorded | 69ba31900636e5bf4afc0884 | | QuaRot | 2327 | Not recorded | 69ba31900636e5bf4afc0884 | | SpinQuant | 2321 | Not recorded | 69ba31900636e5bf4afc0884 | | SmoothQuant | 2317 | Not recorded | 69ba31900636e5bf4afc0884 | | QuaRot | 2296 | Not recorded | 69ba31900636e5bf4afc0884 | | RTN | 2294 | Not recorded | 69ba31900636e5bf4afc0884 | | FlatQuant | 2293 | Not recorded | 69ba31900636e5bf4afc0884 | | GPTQ | 2286 | Not recorded | 69ba31900636e5bf4afc0884 | | BRQ | 2283 | Not recorded | 69ba31900636e5bf4afc0884 | | SmoothQuant | 2283 | Not recorded | 69ba31900636e5bf4afc0884 | | QuaRot | 2280 | Not recorded | 69ba31900636e5bf4afc0884 | | SmoothQuant | 2264 | Not recorded | 69ba31900636e5bf4afc0884 | | SpinQuant | 2248 | Not recorded | 69ba31900636e5bf4afc0884 | | RTN | 2243 | Not recorded | 69ba31900636e5bf4afc0884 | | BRQ | 2236 | Not recorded | 69ba31900636e5bf4afc0884 | | FlatQuant | 2231 | Not recorded | 69ba31900636e5bf4afc0884 | | RTN | 2220 | Not recorded | 69ba31900636e5bf4afc0884 | | SpinQuant | 2217 | Not recorded | 69ba31900636e5bf4afc0884 | | RTN | 2208 | Not recorded | 69ba31900636e5bf4afc0884 | | QuaRot | 2189 | Not recorded | 69ba31900636e5bf4afc0884 | | BRQ | 2147 | Not recorded | 69ba31900636e5bf4afc0884 | | SpinQuant | 1994 | Not recorded | 69ba31900636e5bf4afc0884 | ### BATQuant MME Score: 2386. Result ID: 69ba69d847f34ce924a48156. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV16"}. This record does not assert independent reproduction. ### BF16 MME Score: 2377. Result ID: 69ba69d847f34ce924a4814e. [Reporting paper]() Recorded evaluation configuration: {"Bits":"BF16"}. This record does not assert independent reproduction. ### BATQuant MME Score: 2368. Result ID: 69ba69d847f34ce924a48166. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV8"}. This record does not assert independent reproduction. ### BATQuant MME Score: 2360. Result ID: 69ba69d847f34ce924a4815e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A4KV16"}. This record does not assert independent reproduction. ### FlatQuant MME Score: 2353. Result ID: 69ba69d847f34ce924a48163. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV8"}. This record does not assert independent reproduction. ### FlatQuant MME Score: 2351. Result ID: 69ba69d847f34ce924a48153. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV16"}. This record does not assert independent reproduction. ### SmoothQuant MME Score: 2349. Result ID: 69ba69d847f34ce924a48154. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV16"}. This record does not assert independent reproduction. ### GPTQ MME Score: 2346. Result ID: 69ba69d847f34ce924a48155. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV16"}. This record does not assert independent reproduction. ### GPTQ MME Score: 2340. Result ID: 69ba69d847f34ce924a48165. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV8"}. This record does not assert independent reproduction. ### BATQuant MME Score: 2332. Result ID: 69ba69d847f34ce924a4816e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV4"}. This record does not assert independent reproduction. ### BRQ MME Score: 2329. Result ID: 69ba69d847f34ce924a48152. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV16"}. This record does not assert independent reproduction. ### GPTQ MME Score: 2328. Result ID: 69ba69d847f34ce924a4816d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV4"}. This record does not assert independent reproduction. ### QuaRot MME Score: 2327. Result ID: 69ba69d847f34ce924a48150. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"W4A8KV16"}. This record does not assert independent reproduction. ### SpinQuant MME Score: 2321. Result ID: 69ba69d847f34ce924a48151. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV16"}. This record does not assert independent reproduction. ### SmoothQuant MME Score: 2317. Result ID: 69ba69d847f34ce924a48164. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV8"}. This record does not assert independent reproduction. ### QuaRot MME Score: 2296. Result ID: 69ba69d847f34ce924a48160. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"W4A8KV8"}. This record does not assert independent reproduction. ### RTN MME Score: 2294. Result ID: 69ba69d847f34ce924a4814f. [Reporting paper]() Recorded evaluation configuration: {"Bits":"W4A8KV16"}. This record does not assert independent reproduction. ### FlatQuant MME Score: 2293. Result ID: 69ba69d847f34ce924a4816b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV4"}. This record does not assert independent reproduction. ### GPTQ MME Score: 2286. Result ID: 69ba69d847f34ce924a4815d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A4KV16"}. This record does not assert independent reproduction. ### BRQ MME Score: 2283. Result ID: 69ba69d847f34ce924a48162. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV8"}. This record does not assert independent reproduction. ### SmoothQuant MME Score: 2283. Result ID: 69ba69d847f34ce924a4816c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV4"}. This record does not assert independent reproduction. ### QuaRot MME Score: 2280. Result ID: 69ba69d847f34ce924a48168. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"W4A8KV4"}. This record does not assert independent reproduction. ### SmoothQuant MME Score: 2264. Result ID: 69ba69d847f34ce924a4815c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A4KV16"}. This record does not assert independent reproduction. ### SpinQuant MME Score: 2248. Result ID: 69ba69d847f34ce924a48169. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV4"}. This record does not assert independent reproduction. ### RTN MME Score: 2243. Result ID: 69ba69d847f34ce924a48157. [Reporting paper]() Recorded evaluation configuration: {"Bits":"W4A4KV16"}. This record does not assert independent reproduction. ### BRQ MME Score: 2236. Result ID: 69ba69d847f34ce924a4816a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV4"}. This record does not assert independent reproduction. ### FlatQuant MME Score: 2231. Result ID: 69ba69d847f34ce924a4815b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A4KV16"}. This record does not assert independent reproduction. ### RTN MME Score: 2220. Result ID: 69ba69d847f34ce924a48167. [Reporting paper]() Recorded evaluation configuration: {"Bits":"W4A8KV4"}. This record does not assert independent reproduction. ### SpinQuant MME Score: 2217. Result ID: 69ba69d847f34ce924a48161. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A8KV8"}. This record does not assert independent reproduction. ### RTN MME Score: 2208. Result ID: 69ba69d847f34ce924a4815f. [Reporting paper]() Recorded evaluation configuration: {"Bits":"W4A8KV8"}. This record does not assert independent reproduction. ### QuaRot MME Score: 2189. Result ID: 69ba69d847f34ce924a48158. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Bits":"W4A4KV16"}. This record does not assert independent reproduction. ### BRQ MME Score: 2147. Result ID: 69ba69d847f34ce924a4815a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A4KV16"}. This record does not assert independent reproduction. ### SpinQuant MME Score: 1994. Result ID: 69ba69d847f34ce924a48159. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Bits":"W4A4KV16"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.