Question Answering on TriQA
83.43AccuracyCDKC
Evaluation Results
| Method | Links | |
|---|---|---|
| CDKCBackbone=Qwen2.5-14B-Instruct2026.02 | 83.43 | |
| GRPOBackbone=Qwen2.5-14B-Instruct2026.02 | 82.63 | |
| CoTBackbone=Qwen2.5-14B-Instruct2026.02 | 69.6 | |
| RAGBackbone=Qwen2.5-14B-Instruct2026.02 | 69.1 | |
| CDKCBackbone=Qwen2.5-3B-Instruct2026.02 | 68.23 | |
| GRPOBackbone=Qwen2.5-3B-Instruct2026.02 | 66.7 | |
| CGKEBackbone=Qwen2.5-14B-Instruct2026.02 | 66.47 | |
| Vanilla SFTBackbone=Qwen2.5-14B-Instruct2026.02 | 64.83 | |
| Vanilla LLMBackbone=Qwen2.5-14B-Instruct2026.02 | 63 | |
| RAGBackbone=Qwen2.5-3B-Instruct2026.02 | 61.7 | |
| EMoEBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=6 + 22025.09 | 61.32 | |
| Top-kBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=6 + 22025.09 | 61.1 | |
| Top-kBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=12 + 22025.09 | 60.24 | |
| EMoEBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=12 + 22025.09 | 60.13 | |
| EMoEBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=EMoE, Inference Budget (k')=62025.09 | 59.77 | |
| Top-kBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=Top-k, Inference Budget (k')=62025.09 | 59.53 | |
| EMoEBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=EMoE, Inference Budget (k')=42025.09 | 59.24 | |
| EMoEBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=3 + 22025.09 | 59.22 | |
| Top-kBase Model=ERNIE-4.5-21B-A3B, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=3 + 22025.09 | 59.1 | |
| Top-kBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=Top-k, Inference Budget (k')=42025.09 | 59.06 | |
| EMoEBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=EMoE, Inference Budget (k')=22025.09 | 58.2 | |
| Top-kBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=Top-k, Inference Budget (k')=22025.09 | 57.62 | |
| EMoEBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=EMoE, Inference Budget (k')=12025.09 | 54.79 | |
| Top-kBase Model=LoRAMoE, Trained Activated Experts=2, MoE Routing Method=Top-k, Inference Budget (k')=12025.09 | 54.32 | |
| Llama-3.1Size=8B, Type=AR2026.01 | 52.1 | |
| EMoEBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=12 + 22025.09 | 52.07 | |
| EMoEBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=6 + 22025.09 | 51.58 | |
| CoTBackbone=Qwen2.5-3B-Instruct2026.02 | 49.47 | |
| CGKEBackbone=Qwen2.5-3B-Instruct2026.02 | 48.47 | |
| EMoEBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=EMoE, Inference Budget (k')=3 + 22025.09 | 47.81 | |
| Vanilla SFTBackbone=Qwen2.5-3B-Instruct2026.02 | 47.37 | |
| Top-kBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=6 + 22025.09 | 46.86 | |
| Top-kBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=12 + 22025.09 | 46.38 | |
| Vanilla LLMBackbone=Qwen2.5-3B-Instruct2026.02 | 45.07 | |
| Top-kBase Model=DeepSeek-V2-Lite, Trained Activated Experts=6+2, MoE Routing Method=Top-k, Inference Budget (k')=3 + 22025.09 | 41.36 | |
| EMoEBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=EMoE, Inference Budget (k')=162025.09 | 41.27 | |
| EMoEBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=EMoE, Inference Budget (k')=82025.09 | 39.64 | |
| Top-kBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=Top-k, Inference Budget (k')=162025.09 | 39.1 | |
| Top-kBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=Top-k, Inference Budget (k')=82025.09 | 38.98 | |
| EMoEBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=EMoE, Inference Budget (k')=42025.09 | 34.84 | |
| Top-kBase Model=OLMoE-1B-7B-0924, Trained Activated Experts=8, MoE Routing Method=Top-k, Inference Budget (k')=42025.09 | 32.04 | |
| A3Size=8B, Type=A32026.01 | 19.4 | |
| DiffuLlamaSize=7B, Type=DD2026.01 | 18.5 | |
| DreamSize=7B, Type=DD2026.01 | 18.3 | |
| A3Size=3B, Type=A32026.01 | 15.9 | |
| A3Size=1B, Type=A32026.01 | 10.2 | |
| PlaidSize=1B, Type=CD2026.01 | 1.2 |