List your company
ResearchFormerly WizWand

Chart Question Answering on ChartQA (Accuracy)

90.49AccuracyMetaForge
63.834870.754977.67584.5951Sep 28, 2025Nov 13, 2025Dec 29, 2025Feb 14, 2026Apr 1, 2026May 17, 2026Jul 3, 2026
Updated 3mo ago

Evaluation results

MethodLinks
MetaForgeTool Setting=w/ IID Tools2026.0690.49
MetaForgeTool Setting=w/ OOD Tools2026.0690.18
BF16Model=Kimi-VL-A3B-Instruct, Avg-Bit=BF162026.0689.44
SigLIP2-gTokens per frame/tile=256, TTFT (ms)=517.6, LLM Backbone=9B Nemotron2026.0789.4
RADIO1DTokens per frame/tile=192, TTFT (ms)=380.4, LLM Backbone=9B Nemotron2026.0789.2
RADIO1DTokens per frame/tile=224, TTFT (ms)=410.2, LLM Backbone=9B Nemotron2026.0789.2
C-RADIOv4-HTokens per frame/tile=256, TTFT (ms)=468.2, LLM Backbone=9B Nemotron2026.0789.1
RADIO1DTokens per frame/tile=256, TTFT (ms)=452.8, LLM Backbone=9B Nemotron2026.0789.1
SigLIP2-SO400mTokens per frame/tile=256, TTFT (ms)=440.5, LLM Backbone=9B Nemotron2026.0788.8
RADIO1DTokens per frame/tile=128, TTFT (ms)=373.2, LLM Backbone=9B Nemotron2026.0788.6
RADIO1DTokens per frame/tile=64, TTFT (ms)=346.1, LLM Backbone=9B Nemotron2026.0787.9
FP16 (Baseline)Backbone=Qwen 2.5 VL 7B, Avg. Bit width=162025.0987.3
MODEModel=Kimi-VL-A3B-Instruct, Avg-Bit=W32026.0687.26
InternVL3Size=8B2026.0686.6
LLaVA-OneVision-1.5Size=8B2026.0686.5
GPT-5-highSize=-2026.0686.2
RADIO1DTokens per frame/tile=32, TTFT (ms)=335.1, LLM Backbone=9B Nemotron2026.0785.8
VEQ-MEModel=Kimi-VL-A3B-Instruct, Avg-Bit=W32026.0685.64
BF16Model=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=BF162026.0685.2
AWQBackbone=Qwen 2.5 VL 7B, Avg. Bit width=42025.0984.5
MC-MoEModel=Kimi-VL-A3B-Instruct, Avg-Bit=W32026.0684.26
GPTQBackbone=Qwen 2.5 VL 7B, Avg. Bit width=42025.0984.1
MoEQuantModel=Kimi-VL-A3B-Instruct, Avg-Bit=W32026.0683.77
Claude-Opus-4.1Size=-2026.0683.3
MBQModel=Kimi-VL-A3B-Instruct, Avg-Bit=W32026.0683.13
GreedyModel=Qwen2.5-VL (3B)2026.0483.12
MODEModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W32026.0683
Speed-QModel=Kimi-VL-A3B-Instruct, Avg-Bit=W32026.0682.97
OmniQuantBackbone=Qwen 2.5 VL 7B, Avg. Bit width=42025.0982.9
RADIO1DTokens per frame/tile=8, TTFT (ms)=329.7, LLM Backbone=9B Nemotron2026.0782.8
GreedyBackbone=Qwen2.5-VL (7B)2026.0482.13
GPTQModel=Kimi-VL-A3B-Instruct, Avg-Bit=W32026.0682.04
Top-p (nucleus)Model=Qwen2.5-VL (3B), tau=0.72026.0481.13
VEQ-MEModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W32026.0681.12
ε-SamplingTemperature (τ)=0.7, Backbone=Qwen2.5-VL (7B)2026.0480.16
η-SamplingTemperature (τ)=0.7, Backbone=Qwen2.5-VL (7B)2026.0480.16
Top-p (nucleus)Temperature (τ)=0.7, Backbone=Qwen2.5-VL (7B)2026.0480
MC-MoEModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W32026.0679.94
ε-SamplingModel=Qwen2.5-VL (3B), tau=0.72026.0479.91
Top-p (nucleus)Temperature (τ)=1.0, Backbone=Qwen2.5-VL (7B)2026.0479.69
η-SamplingModel=Qwen2.5-VL (3B), tau=0.72026.0479.66
Min-pModel=Qwen2.5-VL (3B), tau=0.72026.0479.62
Top-kModel=Qwen2.5-VL (3B), tau=0.72026.0479.61
Temp' OnlyModel=Qwen2.5-VL (3B), tau=0.72026.0479.58
Temp’ OnlyTemperature (τ)=0.7, Backbone=Qwen2.5-VL (7B)2026.0478.96
Top-kTemperature (τ)=0.7, Backbone=Qwen2.5-VL (7B)2026.0478.96
Top-p (nucleus)Model=Qwen2.5-VL (3B), tau=1.02026.0478.93
η-SamplingTemperature (τ)=1.0, Backbone=Qwen2.5-VL (7B)2026.0478.73
SMMDBackbone=Qwen2.5-VL, Parameters=7B2026.0678.38
ε-SamplingTemperature (τ)=1.0, Backbone=Qwen2.5-VL (7B)2026.0478.18
Temp’ OnlyTemperature (τ)=1.0, Backbone=Qwen2.5-VL (7B)2026.0477.97
Top-kTemperature (τ)=1.0, Backbone=Qwen2.5-VL (7B)2026.0477.92
NTILBackbone=Qwen2.5-VL, Parameters=7B2026.0677.08
CEBackbone=Qwen2.5-VL, Parameters=7B2026.0677.02
NTLBackbone=Qwen2.5-VL, Parameters=7B2026.0676.92
ε-SamplingModel=Qwen2.5-VL (3B), tau=1.02026.0476.53
η-SamplingModel=Qwen2.5-VL (3B), tau=1.02026.0476.46
Temp' OnlyModel=Qwen2.5-VL (3B), tau=1.02026.0476.14
Top-kModel=Qwen2.5-VL (3B), tau=1.02026.0476.11
Min-pModel=Qwen2.5-VL (3B), tau=1.02026.0476.09
GCEBackbone=Qwen2.5-VL, Parameters=7B2026.0675.87
MODEModel=Kimi-VL-A3B-Instruct, Avg-Bit=W22026.0675.2
CoLTSize=8B, Backbone=Qwen3-VL-8B2026.0674.7
SMMDBackbone=Ministral-3, Parameters=3B2026.0673.21
Speed-QModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W32026.0673.2
MODEModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W22026.0673.11
NTILBackbone=Qwen2.5-VL, Parameters=3B2026.0672.95
GCEBackbone=Ministral-3, Parameters=3B2026.0672.32
SMMDBackbone=Qwen2.5-VL, Parameters=3B2026.0672.21
NTLBackbone=Qwen2.5-VL, Parameters=3B2026.0672.01
NTILBackbone=Ministral-3, Parameters=3B2026.0671.95
CEBackbone=Ministral-3, Parameters=3B2026.0671.64
Min-pTemperature (τ)=1.0, Backbone=Qwen2.5-VL (7B)2026.0471.56
Min-pTemperature (τ)=2.0, Backbone=Qwen2.5-VL (7B)2026.0471.51
CEBackbone=Qwen2.5-VL, Parameters=3B2026.0671.44
Min-pTemperature (τ)=0.7, Backbone=Qwen2.5-VL (7B)2026.0471.41
MoEQuantModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W32026.0671.4
GCEBackbone=Qwen2.5-VL, Parameters=3B2026.0670.44
Qwen2.5-VL-3Bvariant=MoDoMoDo2025.1070.4
RECAPbackbone=Qwen2.5-VL-3B2025.1070.4
Qwen2.5-VL-3B-Instruct-GRPO-deepmath2025.1070.24
ε-SamplingTemperature (τ)=2.0, Backbone=Qwen2.5-VL (7B)2026.0469.79
Qwen2.5-VL-3Bvariant=Uniform2025.1069.68
NTLBackbone=Ministral-3, Parameters=3B2026.0669.66
LUQBackbone=Qwen 2.5 VL 7B, Avg. Bit width=2.75, Quantized Layers=12 layer2025.0968.6
RADIO1DTokens per frame/tile=1, TTFT (ms)=327.0, LLM Backbone=9B Nemotron2026.0768.2
MBQModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W32026.0668.16
vision-grpo-qwen-2.5-vl-3b2025.1067.8
Top-p (nucleus)Temperature (τ)=2.0, Backbone=Qwen2.5-VL (7B)2026.0467.6
LVRSize=7B2026.0667.2
VEQ-MEModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W22026.0667.16
SIM-CoTSize=8B, Backbone=Qwen3-VL-8B2026.0666.7
MC-MoEModel=Kimi-VL-A3B-Instruct, Avg-Bit=W22026.0666.49
η-SamplingTemperature (τ)=2.0, Backbone=Qwen2.5-VL (7B)2026.0466.2
LaViDa-O + RL (Global + Factorized)Arch=D-Diff2026.0666.16
GPTQModel=Qwen3-VL-30B-A3B-Instruct, Avg-Bit=W32026.0666.16
Qwen3-VL (Textual reasoning)Size=8B2026.0665.1
GCEBackbone=Llama-3.2-Vision, Parameters=11B2026.0664.91
SMMDBackbone=Llama-3.2-Vision, Parameters=11B2026.0664.91
CEBackbone=Llama-3.2-Vision, Parameters=11B2026.0664.86