Chart Question Answering on ChartQA (Accuracy)
90.49AccuracyMetaForge
Evaluation Results
| Method | Links | |
|---|---|---|
| MetaForgeTool Setting=w/ IID Tools2026.06 | 90.49 | |
| MetaForgeTool Setting=w/ OOD Tools2026.06 | 90.18 | |
| BF16Model=Kimi-VL-A3B-Instruct, Avg-Bit=BF162026.06 | 89.44 | |
| SigLIP2-gTokens per frame/tile=256, TTFT (ms)=517.6, LLM Backbone=9B Nemotron2026.07 | 89.4 | |
| RADIO1DTokens per frame/tile=192, TTFT (ms)=380.4, LLM Backbone=9B Nemotron2026.07 | 89.2 | |
| RADIO1DTokens per frame/tile=224, TTFT (ms)=410.2, LLM Backbone=9B Nemotron2026.07 | 89.2 | |
| C-RADIOv4-HTokens per frame/tile=256, TTFT (ms)=468.2, LLM Backbone=9B Nemotron2026.07 | 89.1 | |
| RADIO1DTokens per frame/tile=256, TTFT (ms)=452.8, LLM Backbone=9B Nemotron2026.07 | 89.1 | |
| SigLIP2-SO400mTokens per frame/tile=256, TTFT (ms)=440.5, LLM Backbone=9B Nemotron2026.07 | 88.8 | |
| RADIO1DTokens per frame/tile=128, TTFT (ms)=373.2, LLM Backbone=9B Nemotron2026.07 | 88.6 | |
| RADIO1DTokens per frame/tile=64, TTFT (ms)=346.1, LLM Backbone=9B Nemotron2026.07 | 87.9 | |
| FP16 (Baseline)Backbone=Qwen 2.5 VL 7B, Avg. Bit width=162025.09 | 87.3 | |
| MODEModel=Kimi-VL-A3B-Instruct, Avg-Bit=W32026.06 | 87.26 | |
| InternVL3Size=8B2026.06 | 86.6 | |
| LLaVA-OneVision-1.5Size=8B2026.06 | 86.5 | |
| GPT-5-highSize=-2026.06 | 86.2 | |
| RADIO1DTokens per frame/tile=32, TTFT (ms)=335.1, LLM Backbone=9B Nemotron2026.07 | 85.8 | |
| VEQ-MEModel=Kimi-VL-A3B-Instruct, Avg-Bit=W32026.06 | 85.64 | |
| BF16Model=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=BF162026.06 | 85.2 | |
| AWQBackbone=Qwen 2.5 VL 7B, Avg. Bit width=42025.09 | 84.5 | |
| MC-MoEModel=Kimi-VL-A3B-Instruct, Avg-Bit=W32026.06 | 84.26 | |
| GPTQBackbone=Qwen 2.5 VL 7B, Avg. Bit width=42025.09 | 84.1 | |
| MoEQuantModel=Kimi-VL-A3B-Instruct, Avg-Bit=W32026.06 | 83.77 | |
| Claude-Opus-4.1Size=-2026.06 | 83.3 | |
| MBQModel=Kimi-VL-A3B-Instruct, Avg-Bit=W32026.06 | 83.13 | |
| GreedyModel=Qwen2.5-VL (3B)2026.04 | 83.12 | |
| MODEModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W32026.06 | 83 | |
| Speed-QModel=Kimi-VL-A3B-Instruct, Avg-Bit=W32026.06 | 82.97 | |
| OmniQuantBackbone=Qwen 2.5 VL 7B, Avg. Bit width=42025.09 | 82.9 | |
| RADIO1DTokens per frame/tile=8, TTFT (ms)=329.7, LLM Backbone=9B Nemotron2026.07 | 82.8 | |
| GreedyBackbone=Qwen2.5-VL (7B)2026.04 | 82.13 | |
| GPTQModel=Kimi-VL-A3B-Instruct, Avg-Bit=W32026.06 | 82.04 | |
| Top-p (nucleus)Model=Qwen2.5-VL (3B), tau=0.72026.04 | 81.13 | |
| VEQ-MEModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W32026.06 | 81.12 | |
| ε-SamplingTemperature (τ)=0.7, Backbone=Qwen2.5-VL (7B)2026.04 | 80.16 | |
| η-SamplingTemperature (τ)=0.7, Backbone=Qwen2.5-VL (7B)2026.04 | 80.16 | |
| Top-p (nucleus)Temperature (τ)=0.7, Backbone=Qwen2.5-VL (7B)2026.04 | 80 | |
| MC-MoEModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W32026.06 | 79.94 | |
| ε-SamplingModel=Qwen2.5-VL (3B), tau=0.72026.04 | 79.91 | |
| Top-p (nucleus)Temperature (τ)=1.0, Backbone=Qwen2.5-VL (7B)2026.04 | 79.69 | |
| η-SamplingModel=Qwen2.5-VL (3B), tau=0.72026.04 | 79.66 | |
| Min-pModel=Qwen2.5-VL (3B), tau=0.72026.04 | 79.62 | |
| Top-kModel=Qwen2.5-VL (3B), tau=0.72026.04 | 79.61 | |
| Temp' OnlyModel=Qwen2.5-VL (3B), tau=0.72026.04 | 79.58 | |
| Temp’ OnlyTemperature (τ)=0.7, Backbone=Qwen2.5-VL (7B)2026.04 | 78.96 | |
| Top-kTemperature (τ)=0.7, Backbone=Qwen2.5-VL (7B)2026.04 | 78.96 | |
| Top-p (nucleus)Model=Qwen2.5-VL (3B), tau=1.02026.04 | 78.93 | |
| η-SamplingTemperature (τ)=1.0, Backbone=Qwen2.5-VL (7B)2026.04 | 78.73 | |
| SMMDBackbone=Qwen2.5-VL, Parameters=7B2026.06 | 78.38 | |
| ε-SamplingTemperature (τ)=1.0, Backbone=Qwen2.5-VL (7B)2026.04 | 78.18 | |
| Temp’ OnlyTemperature (τ)=1.0, Backbone=Qwen2.5-VL (7B)2026.04 | 77.97 | |
| Top-kTemperature (τ)=1.0, Backbone=Qwen2.5-VL (7B)2026.04 | 77.92 | |
| NTILBackbone=Qwen2.5-VL, Parameters=7B2026.06 | 77.08 | |
| CEBackbone=Qwen2.5-VL, Parameters=7B2026.06 | 77.02 | |
| NTLBackbone=Qwen2.5-VL, Parameters=7B2026.06 | 76.92 | |
| ε-SamplingModel=Qwen2.5-VL (3B), tau=1.02026.04 | 76.53 | |
| η-SamplingModel=Qwen2.5-VL (3B), tau=1.02026.04 | 76.46 | |
| Temp' OnlyModel=Qwen2.5-VL (3B), tau=1.02026.04 | 76.14 | |
| Top-kModel=Qwen2.5-VL (3B), tau=1.02026.04 | 76.11 | |
| Min-pModel=Qwen2.5-VL (3B), tau=1.02026.04 | 76.09 | |
| GCEBackbone=Qwen2.5-VL, Parameters=7B2026.06 | 75.87 | |
| MODEModel=Kimi-VL-A3B-Instruct, Avg-Bit=W22026.06 | 75.2 | |
| CoLTSize=8B, Backbone=Qwen3-VL-8B2026.06 | 74.7 | |
| SMMDBackbone=Ministral-3, Parameters=3B2026.06 | 73.21 | |
| Speed-QModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W32026.06 | 73.2 | |
| MODEModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W22026.06 | 73.11 | |
| NTILBackbone=Qwen2.5-VL, Parameters=3B2026.06 | 72.95 | |
| GCEBackbone=Ministral-3, Parameters=3B2026.06 | 72.32 | |
| SMMDBackbone=Qwen2.5-VL, Parameters=3B2026.06 | 72.21 | |
| NTLBackbone=Qwen2.5-VL, Parameters=3B2026.06 | 72.01 | |
| NTILBackbone=Ministral-3, Parameters=3B2026.06 | 71.95 | |
| CEBackbone=Ministral-3, Parameters=3B2026.06 | 71.64 | |
| Min-pTemperature (τ)=1.0, Backbone=Qwen2.5-VL (7B)2026.04 | 71.56 | |
| Min-pTemperature (τ)=2.0, Backbone=Qwen2.5-VL (7B)2026.04 | 71.51 | |
| CEBackbone=Qwen2.5-VL, Parameters=3B2026.06 | 71.44 | |
| Min-pTemperature (τ)=0.7, Backbone=Qwen2.5-VL (7B)2026.04 | 71.41 | |
| MoEQuantModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W32026.06 | 71.4 | |
| GCEBackbone=Qwen2.5-VL, Parameters=3B2026.06 | 70.44 | |
| Qwen2.5-VL-3Bvariant=MoDoMoDo2025.10 | 70.4 | |
| RECAPbackbone=Qwen2.5-VL-3B2025.10 | 70.4 | |
| Qwen2.5-VL-3B-Instruct-GRPO-deepmath2025.10 | 70.24 | |
| ε-SamplingTemperature (τ)=2.0, Backbone=Qwen2.5-VL (7B)2026.04 | 69.79 | |
| Qwen2.5-VL-3Bvariant=Uniform2025.10 | 69.68 | |
| NTLBackbone=Ministral-3, Parameters=3B2026.06 | 69.66 | |
| LUQBackbone=Qwen 2.5 VL 7B, Avg. Bit width=2.75, Quantized Layers=12 layer2025.09 | 68.6 | |
| RADIO1DTokens per frame/tile=1, TTFT (ms)=327.0, LLM Backbone=9B Nemotron2026.07 | 68.2 | |
| MBQModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W32026.06 | 68.16 | |
| vision-grpo-qwen-2.5-vl-3b2025.10 | 67.8 | |
| Top-p (nucleus)Temperature (τ)=2.0, Backbone=Qwen2.5-VL (7B)2026.04 | 67.6 | |
| LVRSize=7B2026.06 | 67.2 | |
| VEQ-MEModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W22026.06 | 67.16 | |
| SIM-CoTSize=8B, Backbone=Qwen3-VL-8B2026.06 | 66.7 | |
| MC-MoEModel=Kimi-VL-A3B-Instruct, Avg-Bit=W22026.06 | 66.49 | |
| η-SamplingTemperature (τ)=2.0, Backbone=Qwen2.5-VL (7B)2026.04 | 66.2 | |
| LaViDa-O + RL (Global + Factorized)Arch=D-Diff2026.06 | 66.16 | |
| GPTQModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W32026.06 | 66.16 | |
| Qwen3-VL (Textual reasoning)Size=8B2026.06 | 65.1 | |
| GCEBackbone=Llama-3.2-Vision, Parameters=11B2026.06 | 64.91 | |
| SMMDBackbone=Llama-3.2-Vision, Parameters=11B2026.06 | 64.91 | |
| CEBackbone=Llama-3.2-Vision, Parameters=11B2026.06 | 64.86 |