Multi-modal Understanding on General Multi-modal Evaluation Suite (test val)
77.7VQAv2 AccuracyMoE-LLaVA-Phi DeRS-SM
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| MoE-LLaVA-Phi DeRS-SMMoE Model Backbone=Phi-2-2.7B, Upcycling Method=DeRS-SM, Added Params=1.11M2025.03 | 77.7 | 61.3 | 42.4 | 69 | 51.7 | 87.3 | 65.5 | 33.8 | 61.1 | |
| MoE-LLaVA-Phi DeRS-LMMoE Model Backbone=Phi-2-2.7B, Upcycling Method=DeRS-LM, Added Params=2.42M2025.03 | 77.6 | 61.3 | 42 | 69.1 | 51.8 | 87.4 | 66.3 | 32.6 | 61 | |
| MoE-LLaVA-Phi VanillaMoE Model Backbone=Phi-2-2.7B, Upcycling Method=Vanilla, Added Params=2.52B2025.03 | 77.5 | 61.4 | 42.7 | 68.6 | 50.8 | 86.9 | 66 | 32.4 | 60.8 | |
| MoE-LLaVA-StableLM DeRS-LMMoE Model Backbone=StableLM-2-1.6B, Upcycling Method=DeRS-LM, Added Params=1.20M2025.03 | 76.6 | 60.6 | 34.4 | 62.3 | 50 | 87.1 | 60.1 | 28.6 | 57.5 | |
| MoE-LLaVA-StableLM DeRS-SMMoE Model Backbone=StableLM-2-1.6B, Upcycling Method=DeRS-SM, Added Params=0.26M2025.03 | 76.5 | 60.7 | 34.9 | 62.9 | 50.2 | 87.3 | 60.4 | 28.4 | 57.7 | |
| MoE-LLaVA-StableLM VanillaMoE Model Backbone=StableLM-2-1.6B, Upcycling Method=Vanilla, Added Params=1.24B2025.03 | 76.3 | 60.6 | 34.6 | 62.7 | 50.2 | 86.9 | 60.5 | 27.1 | 57.4 | |
| MoE-LLaVA-Qwen VanillaMoE Model Backbone=Qwen-1.8B, Upcycling Method=Vanilla, Added Params=1.22B2025.03 | 76.2 | 61.2 | 31.8 | 62.4 | 48.1 | 87.5 | 60.4 | 24.4 | 56.5 | |
| MoE-LLaVA-Qwen DeRS-SMMoE Model Backbone=Qwen-1.8B, Upcycling Method=DeRS-SM, Added Params=0.26M2025.03 | 76.2 | 61.2 | 31 | 62.2 | 47.8 | 87.8 | 60 | 23.7 | 56.2 | |
| MoE-LLaVA-Qwen DeRS-LMMoE Model Backbone=Qwen-1.8B, Upcycling Method=DeRS-LM, Added Params=1.19M2025.03 | 76.2 | 61.1 | 31.2 | 62.1 | 47.9 | 87.8 | 60 | 24.2 | 56.3 |