Visual Question Answering on Cambrian Benchmarks
77.7General AverageHeRA
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| HeRABackbone LLM=Qwen3-14B, HeRA Regularization=true2026.06 | 77.7 | 52.6 | 47.8 | 62.5 | 58 | 52 | 51.8 | 70.2 | 58.9 | |
| HeRABackbone LLM=Qwen2.5-14B, HeRA Regularization=true2026.06 | 77.4 | 52.8 | 49.3 | 60.1 | 52 | 55.2 | 52.4 | 71.6 | 58.3 | |
| Qwen3-14BBackbone LLM=Qwen3-14B, HeRA Regularization=false2026.06 | 77.4 | 52.8 | 46.1 | 60.3 | 57.3 | 52.6 | 50.3 | 70.8 | 58.2 | |
| HeRABackbone LLM=Qwen3-8B, HeRA Regularization=true2026.06 | 76.9 | 51.1 | 47.6 | 60.1 | 58 | 55.5 | 50.3 | 73.5 | 59.5 | |
| HeRAAlignment Strategy=Head-Wise Representation Alignment2026.06 | 76.9 | 51.1 | 47.6 | 60.1 | 58 | 55.5 | 50.3 | 73.5 | 59.5 | |
| JARVISAlignment Strategy=Representation Alignment2026.06 | 76.8 | 49.9 | 46.2 | 59.2 | 54.7 | 54.2 | 55.5 | 69.9 | 58.7 | |
| HeRABackbone LLM=Qwen2.5-7B, HeRA Regularization=true2026.06 | 76.5 | 50.5 | 48.6 | 61.3 | 54 | 50.2 | 50.3 | 71.3 | 57.4 | |
| CMARAlignment Strategy=Representation Alignment2026.06 | 76.4 | 51 | 46 | 58.8 | 50 | 52.1 | 52.9 | 70.9 | 56.9 | |
| Qwen2.5-7BBackbone LLM=Qwen2.5-7B, HeRA Regularization=false2026.06 | 76.2 | 50.2 | 47.9 | 59.6 | 51.3 | 51.7 | 50.8 | 70.2 | 56.7 | |
| HeRABackbone LLM=Qwen3-4B, HeRA Regularization=true2026.06 | 76 | 50.1 | 44.5 | 61.3 | 56 | 53.7 | 52.4 | 69.1 | 58.5 | |
| Qwen3-4BBackbone LLM=Qwen3-4B, HeRA Regularization=false2026.06 | 75.6 | 49.6 | 43.8 | 59.9 | 48 | 55.1 | 50.3 | 68.3 | 56.3 | |
| Qwen2.5-14BBackbone LLM=Qwen2.5-14B, HeRA Regularization=false2026.06 | 75.6 | 50.7 | 44.8 | 60.1 | 47.3 | 52.7 | 46.6 | 67.9 | 54.9 | |
| Qwen3-8BBackbone LLM=Qwen3-8B, HeRA Regularization=false2026.06 | 74.7 | 49.7 | 43.5 | 59.2 | 49.3 | 52.2 | 50.8 | 67.8 | 55.9 | |
| LLaVA (Standard Training Recipe)Alignment Strategy=Baseline2026.06 | 74.7 | 49.7 | 43.5 | 59.2 | 49.3 | 52.2 | 50.8 | 67.8 | 55.9 | |
| HeRABackbone LLM=LLama3-8B, HeRA Regularization=true2026.06 | 74.6 | 46.3 | 44.7 | 60.4 | 46.7 | 50.2 | 51.8 | 66.4 | 55.1 | |
| ROSSAlignment Strategy=Representation Alignment2026.06 | 74.6 | 49.4 | 44 | 59.8 | 50.3 | 52.2 | 49.2 | 69.7 | 56.2 | |
| HeRABackbone LLM=Qwen2.5-3B, HeRA Regularization=true2026.06 | 74.5 | 47.5 | 43.8 | 56.3 | 48 | 49.1 | 51.3 | 59.6 | 52.9 | |
| VIRALAlignment Strategy=Representation Alignment2026.06 | 73.8 | 49.3 | 43.6 | 57.6 | 46.7 | 51.3 | 46.1 | 69.3 | 54.2 | |
| HeRABackbone LLM=Vicuna-13B, HeRA Regularization=true2026.06 | 73.6 | 45.7 | 47.6 | 57.3 | 44 | 52.3 | 49.2 | 66.5 | 53.9 | |
| Qwen2.5-3BBackbone LLM=Qwen2.5-3B, HeRA Regularization=false2026.06 | 73.5 | 46.7 | 42.2 | 55.2 | 46 | 46.8 | 44.5 | 60.2 | 50.5 | |
| Vicuna-13BBackbone LLM=Vicuna-13B, HeRA Regularization=false2026.06 | 73.4 | 45.5 | 47.7 | 58.7 | 44.7 | 50.6 | 46.1 | 63.7 | 52.7 | |
| LLama3-8BBackbone LLM=LLama3-8B, HeRA Regularization=false2026.06 | 73.3 | 45 | 43 | 60.1 | 46 | 49.2 | 44 | 69.5 | 53.8 | |
| Vicuna-7BBackbone LLM=Vicuna-7B, HeRA Regularization=false2026.06 | 72.2 | 44.3 | 45.7 | 56.5 | 38.7 | 46.8 | 44.5 | 62.1 | 49.7 | |
| HeRABackbone LLM=Vicuna-7B, HeRA Regularization=true2026.06 | 72.1 | 44.5 | 45.7 | 57.8 | 42.7 | 47.9 | 49.7 | 61.9 | 52 |