Continual Multimodal Instruction Tuning on CoIN/VQA Composite Benchmark
68.71AccuracyUpper bound
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Upper boundBackbone=Janus-Pro-1B2025.12 | 68.71 | — | — | — | — | — | — | — | |
| Upper bound2025.12 | 57.32 | — | — | — | — | — | — | — | |
| Upper boundBackbone=Chameleon [3], Evaluation Stage=Individually fine-tuning a model on each task2025.12 | 55.25 | 73.27 | 40.74 | 91.88 | 36.56 | 44.15 | 44.9 | — | |
| MoDEBackbone=Janus-Pro-1B2025.12 | 53.02 | — | — | — | — | — | — | 14.48 | |
| CL-MoEBackbone=Janus-Pro-1B2025.12 | 43.18 | — | — | — | — | — | — | 21.96 | |
| Seq LoRABackbone=Janus-Pro-1B2025.12 | 37.65 | — | — | — | — | — | — | 27.37 | |
| MoDEBackbone=Chameleon [3], Evaluation Stage=Final accuracy after completing all tasks2025.12 | 37.05 | 67.87 | 33.36 | 9.54 | 33.6 | 41.91 | 36.64 | 18.25 | |
| CL-MoEBackbone=Chameleon [3], Evaluation Stage=Final accuracy after completing all tasks2025.12 | 36.96 | 67.32 | 32.82 | 9.12 | 33.29 | 43.2 | 36.22 | 19.12 | |
| Zero-shotBackbone=Janus-Pro-1B2025.12 | 35.54 | — | — | — | — | — | — | — | |
| MoELoRABackbone=Chameleon [3], Evaluation Stage=Final accuracy after completing all tasks2025.12 | 34.14 | 65.89 | 22.36 | 6.71 | 30.22 | 42.55 | 37.13 | 24.08 | |
| MoDE2025.12 | 33.47 | — | — | — | — | — | — | 25.99 | |
| MoELoRA2025.12 | 33.01 | — | — | — | — | — | — | 30.77 | |
| CL-MoE2025.12 | 32.86 | — | — | — | — | — | — | 30.95 | |
| Model tailor2025.12 | 32.62 | — | — | — | — | — | — | 27.66 | |
| DualPrompt2025.12 | 31.92 | — | — | — | — | — | — | 6.82 | |
| Seq LoRABackbone=Chameleon [3], Evaluation Stage=Final accuracy after completing all tasks2025.12 | 29.82 | 38.67 | 33.18 | 15.09 | 34.99 | 39.34 | 18.62 | 24.64 | |
| Model TailorBackbone=Janus-Pro-1B2025.12 | 29.13 | — | — | — | — | — | — | 32.01 | |
| Model tailorBackbone=Chameleon [3], Evaluation Stage=Final accuracy after completing all tasks2025.12 | 29.09 | 56.06 | 25.82 | 7.1 | 32.62 | 37.32 | 15.66 | 22.97 | |
| Seq LoRA2025.12 | 28.43 | — | — | — | — | — | — | 35.33 | |
| DualPromptBackbone=Chameleon [3], Evaluation Stage=Final accuracy after completing all tasks2025.12 | 25.47 | 52.45 | 24.12 | 3.82 | 31.44 | 27.16 | 13.82 | 9.57 | |
| Zero-shot2025.12 | 22.48 | — | — | — | — | — | — | — | |
| Zero-shotBackbone=Chameleon [3], Evaluation Stage=Zero-shot2025.12 | 18.73 | 51.52 | 23.49 | 16.53 | 14.22 | 6.64 | 0 | — | |
| CL-MoEBackbone=Chameleon [3], Evaluation Stage=Best accuracy during continual tuning2025.12 | — | 71.35 | 38.82 | 90.08 | 37.37 | 43.73 | 36.22 | — | |
| DualPromptBackbone=Chameleon [3], Evaluation Stage=Best accuracy during continual tuning2025.12 | — | 60.01 | 31.48 | 24.55 | 29.91 | 40.91 | 13.82 | — | |
| MoDEBackbone=Chameleon [3], Evaluation Stage=Best accuracy during continual tuning2025.12 | — | 71.49 | 39 | 85.88 | 36.73 | 44.45 | 36.64 | — | |
| Model tailorBackbone=Chameleon [3], Evaluation Stage=Best accuracy during continual tuning2025.12 | — | 74.9 | 40.04 | 78.22 | 37.35 | 43.25 | 15.66 | — | |
| MoELoRABackbone=Chameleon [3], Evaluation Stage=Best accuracy during continual tuning2025.12 | — | 71.79 | 39.62 | 94.75 | 37.66 | 44.33 | 37.13 | — | |
| Seq LoRABackbone=Chameleon [3], Evaluation Stage=Best accuracy during continual tuning2025.12 | — | 72.43 | 39.34 | 89.41 | 37.93 | 44.38 | 18.62 | — |