Visual Question Answering on TQA
77.5AccuracySPECTRA
Evaluation Results
| Method | Links | |
|---|---|---|
| SPECTRAAgency Type (Agentic vs Non-Agentic)=Agentic, Evaluation Protocol (Zero-shot)=Zero-shot, Backbone Architecture=Qwen2.5-VL, Model Scale (Parameters)=7B, Framework=VERL2026.04 | 77.5 | |
| GPT-4oAgency Type (Agentic vs Non-Agentic)=Non-Agentic, Evaluation Protocol (Zero-shot)=Zero-shot, Backbone Architecture=GPT-4o2026.04 | 77 | |
| Qwen2.5-VLAgency Type (Agentic vs Non-Agentic)=Non-Agentic, Evaluation Protocol (Zero-shot)=Zero-shot, Backbone Architecture=Qwen2.5-VL, Model Scale (Parameters)=7B2026.04 | 74.6 | |
| GPT-4o MiniAgency Type (Agentic vs Non-Agentic)=Non-Agentic, Evaluation Protocol (Zero-shot)=Zero-shot, Backbone Architecture=GPT-4o Mini2026.04 | 73.5 | |
| VERLAgency Type (Agentic vs Non-Agentic)=Agentic, Evaluation Protocol (Zero-shot)=Zero-shot, Backbone Architecture=Qwen2.5-VL, Model Scale (Parameters)=7B, Framework=VERL2026.04 | 73.3 | |
| Gemma-3-ITAgency Type (Agentic vs Non-Agentic)=Non-Agentic, Evaluation Protocol (Zero-shot)=Zero-shot, Backbone Architecture=Gemma-3-IT, Model Scale (Parameters)=4B2026.04 | 73 | |
| HF SmolAgentAgency Type (Agentic vs Non-Agentic)=Agentic, Evaluation Protocol (Zero-shot)=Zero-shot, Backbone Architecture=Qwen2.5-VL, Model Scale (Parameters)=7B, Framework=HF SmolAgent2026.04 | 72.5 | |
| CustomAgency Type (Agentic vs Non-Agentic)=Agentic, Evaluation Protocol (Zero-shot)=Zero-shot, Backbone Architecture=Gemma-3-IT, Model Scale (Parameters)=4B, Framework=Custom2026.04 | 68 | |
| SPECTRAAgency Type (Agentic vs Non-Agentic)=Agentic, Evaluation Protocol (Zero-shot)=Zero-shot, Backbone Architecture=Qwen2.5-VL, Model Scale (Parameters)=3B, Framework=VERL2026.04 | 60 | |
| VERLAgency Type (Agentic vs Non-Agentic)=Agentic, Evaluation Protocol (Zero-shot)=Zero-shot, Backbone Architecture=Qwen2.5-VL, Model Scale (Parameters)=3B, Framework=VERL2026.04 | 56 | |
| Phi-4-MultimodalAgency Type (Agentic vs Non-Agentic)=Non-Agentic, Evaluation Protocol (Zero-shot)=Zero-shot, Backbone Architecture=Phi-4-Multimodal, Model Scale (Parameters)=6B2026.04 | 52 | |
| Qwen2.5-VLAgency Type (Agentic vs Non-Agentic)=Non-Agentic, Evaluation Protocol (Zero-shot)=Zero-shot, Backbone Architecture=Qwen2.5-VL, Model Scale (Parameters)=3B2026.04 | 42.2 | |
| CustomAgency Type (Agentic vs Non-Agentic)=Agentic, Evaluation Protocol (Zero-shot)=Zero-shot, Backbone Architecture=Phi-4-Multimodal, Model Scale (Parameters)=6B, Framework=Custom2026.04 | 20 |