Multiple Choice Question Answering on MMLU (Detailed Subject Accuracy)
74.91MMLU Accuracy (Overall)VideoRAG
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| VideoRAGRetrieval Modality=Video, Generator=Qwen3-VL-8B-Instruct2025.04 | 74.91 | — | — | — | — | — | — | — | — | — | |
| UniversalRAG (T5Gemma 2 270M)Router Type=Trained, Router Model=T5Gemma 2 270M, Generator=Qwen3-VL-8B-Instruct2025.04 | 74.62 | — | — | — | — | — | — | — | — | — | |
| NaïveRetrieval Modality=None (Naïve), Generator=Qwen3-VL-8B-Instruct2025.04 | 74.39 | — | — | — | — | — | — | — | — | — | |
| ParagraphRAGRetrieval Granularity=Paragraph, Generator=Qwen3-VL-8B-Instruct2025.04 | 74.39 | — | — | — | — | — | — | — | — | — | |
| UniversalRAG (Qwen3-VL-2B-Instruct)Router Type=Trained, Router Model=Qwen3-VL-2B-Instruct, Generator=Qwen3-VL-8B-Instruct2025.04 | 74.39 | — | — | — | — | — | — | — | — | — | |
| UniversalRAG (InternVL3.5-1B)Router Type=Trained, Router Model=InternVL3.5-1B, Generator=Qwen3-VL-8B-Instruct2025.04 | 74.39 | — | — | — | — | — | — | — | — | — | |
| OracleRetrieval Type=Oracle, Generator=Qwen3-VL-8B-Instruct2025.04 | 74.39 | — | — | — | — | — | — | — | — | — | |
| UniversalRAG (GPT-5)Router Type=Training-free, Router Model=GPT-5, Generator=Qwen3-VL-8B-Instruct2025.04 | 74.27 | — | — | — | — | — | — | — | — | — | |
| UniversalRAG (Qwen3-VL-8B-Instruct)Router Type=Training-free, Router Model=Qwen3-VL-8B-Instruct, Generator=Qwen3-VL-8B-Instruct2025.04 | 74.09 | — | — | — | — | — | — | — | — | — | |
| ImageRAGRetrieval Modality=Image, Generator=Qwen3-VL-8B-Instruct2025.04 | 73.33 | — | — | — | — | — | — | — | — | — | |
| ClipRAGRetrieval Strategy=CLIP-based, Generator=Qwen3-VL-8B-Instruct2025.04 | 73.33 | — | — | — | — | — | — | — | — | — | |
| TableRAGRetrieval Modality=Table, Generator=Qwen3-VL-8B-Instruct2025.04 | 72.51 | — | — | — | — | — | — | — | — | — | |
| PEcoreGenerator=Qwen3-VL-8B-Instruct2025.04 | 72.11 | — | — | — | — | — | — | — | — | — | |
| VLM2Vec-V2Encoder=VLM2Vec-V2, Generator=Qwen3-VL-8B-Instruct2025.04 | 71.7 | — | — | — | — | — | — | — | — | — | |
| DocumentRAGRetrieval Granularity=Document, Generator=Qwen3-VL-8B-Instruct2025.04 | 71.29 | — | — | — | — | — | — | — | — | — | |
| MultiRAGGenerator=Qwen3-VL-8B-Instruct2025.04 | 70.82 | — | — | — | — | — | — | — | — | — | |
| GMEGenerator=Qwen3-VL-8B-Instruct2025.04 | 70.41 | — | — | — | — | — | — | — | — | — | |
| UniRAGRetrieval Strategy=Unified, Generator=Qwen3-VL-8B-Instruct2025.04 | 70.06 | — | — | — | — | — | — | — | — | — | |
| Spherical SteeringZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.02 | 62.05 | — | — | — | — | — | — | — | — | — | |
| ITIZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.02 | 60.9 | — | — | — | — | — | — | — | — | — | |
| CAAZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.02 | 60.7 | — | — | — | — | — | — | — | — | — | |
| SADI-HEADZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.02 | 60.66 | — | — | — | — | — | — | — | — | — | |
| BaselineZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.02 | 60.6 | — | — | — | — | — | — | — | — | — | |
| PIFModel=LLaMA2-13B2024.06 | 60 | 48 | 42.4 | 68.5 | 65.8 | 58.4 | 52.2 | 65.6 | 62 | 56.8 | |
| BaseModel=Mistral-7B2026.05 | 59.8 | — | — | — | — | — | — | — | — | — | |
| SymbolModel=LLaMA2-13B2024.06 | 59.2 | 46.9 | 42.3 | 68.1 | 64 | 55 | 47.9 | 65.5 | 59.9 | 54.8 | |
| RSCBModel=LLaMA2-13B2024.06 | 58.7 | 47.5 | 41.8 | 68 | 65.2 | 55.8 | 44.3 | 64.6 | 61.8 | 54.3 | |
| SCBModel=LLaMA2-13B2024.06 | 58.6 | 48.7 | 36.2 | 68.5 | 62 | 54.2 | 43.3 | 64.8 | 61.8 | 50.2 | |
| LCDDModel=DeepSeek-8B2026.05 | 55.4 | — | — | — | — | — | — | — | — | — | |
| LCDDModel=Mistral-7B2026.05 | 55.4 | — | — | — | — | — | — | — | — | — | |
| PIFModel=LLaMA2-7B2024.06 | 55 | 44 | 39.5 | 62.7 | 59.5 | 53.1 | 42.4 | 60.9 | 59.3 | 49.5 | |
| SymbolModel=LLaMA2-7B2024.06 | 54.6 | 43.8 | 33.5 | 62.5 | 54.5 | 52.2 | 43.3 | 60.6 | 51.6 | 45.6 | |
| SFTModel=DeepSeek-8B2026.05 | 54.5 | — | — | — | — | — | — | — | — | — | |
| RSCBModel=LLaMA2-7B2024.06 | 54.4 | 43.2 | 39 | 62.3 | 56.8 | 52.1 | 45.5 | 60.5 | 56.8 | 49 | |
| BaseModel=DeepSeek-8B2026.05 | 54.1 | — | — | — | — | — | — | — | — | — | |
| SCBModel=LLaMA2-7B2024.06 | 53.8 | 43.6 | 35 | 62.1 | 55.9 | 50.5 | 45.4 | 60.4 | 54.9 | 47.7 | |
| Full RetrainingBackbone=LLaMA-3.1-Instruct-8B, Number of random seeds=32026.04 | 51.1 | — | — | — | — | — | — | — | — | — | |
| COMPASS-ECDABackbone=LLaMA-3.1-Instruct-8B, Number of random seeds=32026.04 | 50.9 | — | — | — | — | — | — | — | — | — | |
| EWCBackbone=LLaMA-3.1-Instruct-8B, Number of random seeds=32026.04 | 49.4 | — | — | — | — | — | — | — | — | — | |
| Random RehearsalBackbone=LLaMA-3.1-Instruct-8B, Number of random seeds=32026.04 | 49.3 | — | — | — | — | — | — | — | — | — | |
| BaseModel=Vicuna-7B2026.05 | 48.5 | — | — | — | — | — | — | — | — | — | |
| SFTModel=Mistral-7B2026.05 | 47.9 | — | — | — | — | — | — | — | — | — | |
| LCDDModel=Vicuna-7B2026.05 | 47.3 | — | — | — | — | — | — | — | — | — | |
| SFTModel=Vicuna-7B2026.05 | 46.6 | — | — | — | — | — | — | — | — | — | |
| TrigModel=Vicuna-7B2026.05 | 45.8 | — | — | — | — | — | — | — | — | — | |
| SFTModel=Qwen3-0.6B2026.05 | 45.5 | — | — | — | — | — | — | — | — | — | |
| Naive Fine-tuningBackbone=LLaMA-3.1-Instruct-8B, Number of random seeds=32026.04 | 42.8 | — | — | — | — | — | — | — | — | — | |
| TrigModel=Mistral-7B2026.05 | 42.3 | — | — | — | — | — | — | — | — | — | |
| TrigModel=Qwen3-0.6B2026.05 | 41.9 | — | — | — | — | — | — | — | — | — | |
| LCDDModel=Qwen3-0.6B2026.05 | 41.8 | — | — | — | — | — | — | — | — | — | |
| BaseModel=Qwen3-0.6B2026.05 | 40.3 | — | — | — | — | — | — | — | — | — | |
| TrigModel=DeepSeek-8B2026.05 | 37.4 | — | — | — | — | — | — | — | — | — | |
| AdamWBackbone=Llama3-3B-Instruct, Training Mode=Fine-tuning2026.02 | — | 49.92 | — | 65.42 | — | 56.49 | — | — | — | — | |
| AdaSAMBackbone=Llama3-3B-Instruct, Training Mode=Fine-tuning2026.02 | — | 50.49 | — | 65.52 | — | 55.9 | — | — | — | — | |
| Count-guidedModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=2e2026.04 | — | 54.1 | — | 74.9 | — | 58 | — | 70.2 | — | — | |
| Count-guidedModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=4e2026.04 | — | 42 | — | 57.7 | — | 45.7 | — | 45.7 | — | — | |
| Count-guidedModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=2e2026.04 | — | 34.1 | — | 47.4 | — | 38.3 | — | 47.4 | — | — | |
| Count-guidedModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=4e2026.04 | — | 32.9 | — | 42.5 | — | 35.9 | — | 45.6 | — | — | |
| DEKModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=2e2026.04 | — | 60.2 | — | 78.5 | — | 62.3 | — | 73.4 | — | — | |
| DEKModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=4e2026.04 | — | 51.3 | — | 69.7 | — | 57.8 | — | 64.2 | — | — | |
| DEKModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=2e2026.04 | — | 34.9 | — | 47.3 | — | 38.7 | — | 46.9 | — | — | |
| DEKModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=4e2026.04 | — | 33.7 | — | 47 | — | 39.2 | — | 46.6 | — | — | |
| EnumerateModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=2e2026.04 | — | 59.5 | — | 78 | — | 60.4 | — | 73 | — | — | |
| EnumerateModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=4e2026.04 | — | 52.3 | — | 67.2 | — | 53.9 | — | 64.2 | — | — | |
| ESAMBackbone=Llama3-3B-Instruct, Training Mode=Fine-tuning2026.02 | — | 50.02 | — | 64.64 | — | 56.13 | — | — | — | — | |
| GRAPEModel=Mixtral-8x22B, Scope=Global, Number of experts pruned (e)=2e2026.04 | — | 62.7 | — | 80.4 | — | 64.1 | — | 75.3 | — | — | |
| GRAPEModel=Mixtral-8x22B, Scope=Global, Number of experts pruned (e)=4e2026.04 | — | 54.6 | — | 72.9 | — | 58.4 | — | 67.9 | — | — | |
| GRAPEModel=DeepSeek-MoE, Scope=Global, Number of experts pruned (e)=2e2026.04 | — | 35.8 | — | 48.3 | — | 39.7 | — | 50 | — | — | |
| GRAPEModel=DeepSeek-MoE, Scope=Global, Number of experts pruned (e)=4e2026.04 | — | 35.3 | — | 47.6 | — | 39.7 | — | 50 | — | — | |
| OriginalModel=Mixtral-8x22B, Scope=None2026.04 | — | 67.1 | — | 84.1 | — | 68.6 | — | 78.7 | — | — | |
| OriginalModel=DeepSeek-MoE, Scope=None2026.04 | — | 36.1 | — | 47.9 | — | 40.4 | — | 49.5 | — | — | |
| Router-guidedModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=2e2026.04 | — | 24.4 | — | 25.4 | — | 27.3 | — | 27.9 | — | — | |
| Router-guidedModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=4e2026.04 | — | 24 | — | 25.8 | — | 22.7 | — | 23.4 | — | — | |
| Router-guidedModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=2e2026.04 | — | 33.9 | — | 46.4 | — | 38.1 | — | 47.7 | — | — | |
| Router-guidedModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=4e2026.04 | — | 33.6 | — | 41.6 | — | 34.3 | — | 43.1 | — | — | |
| RSTBackbone=Llama3-3B-Instruct, Training Mode=Fine-tuning2026.02 | — | 50.52 | — | 65.58 | — | 55.98 | — | — | — | — | |
| SL-SAMBackbone=Llama3-3B-Instruct, Training Mode=Fine-tuning2026.02 | — | 49.83 | — | 65 | — | 55.73 | — | — | — | — |