Question Answering on OpenBookQA (Accuracy and Token Count)
95.2AccuracyART-max
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ART-maxModel=Qwen2.5-32B-Instruct, Evaluation Protocol=D_test2026.04 | 95.2 | — | |
| ART-meanModel=Qwen2.5-32B-Instruct, Evaluation Protocol=D_test2026.04 | 95.1 | — | |
| VanillaModel=Qwen2.5-32B-Instruct, Evaluation Protocol=D_test2026.04 | 95 | — | |
| VanillaModel=Qwen2.5-14B-Instruct, Evaluation Protocol=D_test2026.04 | 93 | — | |
| ART-meanModel=Qwen2.5-14B-Instruct, Evaluation Protocol=D_test2026.04 | 93 | — | |
| ART-maxModel=Qwen2.5-14B-Instruct, Evaluation Protocol=D_test2026.04 | 92.8 | — | |
| ART-maxModel=Qwen2.5-7B-Instruct, Evaluation Protocol=D_test2026.04 | 87.6 | — | |
| VanillaModel=Qwen2.5-7B-Instruct, Evaluation Protocol=D_test2026.04 | 87.2 | — | |
| ART-meanModel=Qwen2.5-7B-Instruct, Evaluation Protocol=D_test2026.04 | 87.2 | — | |
| ART-maxModel=Llama3.1-8B-Instruct, Evaluation Protocol=D_test2026.04 | 85.4 | — | |
| ART-meanModel=Llama3.1-8B-Instruct, Evaluation Protocol=D_test2026.04 | 85.2 | — | |
| VanillaModel=Qwen2-7B-Instruct, Evaluation Protocol=D_test2026.04 | 85.1 | — | |
| TodyCommAttack Rate=< 50%2026.02 | 84.83 | 428 | |
| ART-meanModel=Qwen2-7B-Instruct, Evaluation Protocol=D_test2026.04 | 84.8 | — | |
| ART-maxModel=Qwen2-7B-Instruct, Evaluation Protocol=D_test2026.04 | 84.6 | — | |
| VanillaModel=Llama3.1-8B-Instruct, Evaluation Protocol=D_test2026.04 | 84.4 | — | |
| VanillaModel=Ministral-8B-Instruct-2410, Evaluation Protocol=D_test2026.04 | 84.3 | — | |
| ART-meanModel=Ministral-8B-Instruct-2410, Evaluation Protocol=D_test2026.04 | 84.3 | — | |
| TodyCommAttack Rate== 50%2026.02 | 84.17 | 448 | |
| ART-maxModel=Ministral-8B-Instruct-2410, Evaluation Protocol=D_test2026.04 | 84.1 | — | |
| TodyCommAttack Rate=> 50%2026.02 | 80.5 | 459 | |
| AgentPruneAttack Rate=< 50%2026.02 | 73.83 | 388 | |
| G-DesignerAttack Rate=< 50%2026.02 | 73.67 | 534 | |
| Complete GraphAttack Rate=< 50%2026.02 | 67.83 | 621 | |
| Random GraphAttack Rate=< 50%2026.02 | 64.33 | 490 | |
| KoCoModel Scale=1.6B Parameters2026.04 | 51.2 | — | |
| URL Prefix (MeCo)Model Scale=1.6B Parameters2026.04 | 50.8 | — | |
| Standard CPTModel Scale=1.6B Parameters2026.04 | 49.6 | — | |
| Data SelectionModel Scale=1.6B Parameters2026.04 | 49 | — | |
| VanillaModel=Llama2-7B-Chat, Evaluation Protocol=D_test2026.04 | 45.9 | — | |
| Mula-7B-A1BModel Type=MoE2026.04 | 45.6 | — | |
| ART-meanModel=Llama2-7B-Chat, Evaluation Protocol=D_test2026.04 | 45.2 | — | |
| ART-maxModel=Llama2-7B-Chat, Evaluation Protocol=D_test2026.04 | 44.9 | — | |
| OLMoE-1B-7B-0924Model Type=MoE2026.04 | 44.8 | — | |
| AgentPruneAttack Rate== 50%2026.02 | 44.17 | 451 | |
| Complete GraphAttack Rate== 50%2026.02 | 42.5 | 655 | |
| Random GraphAttack Rate== 50%2026.02 | 42.17 | 553 | |
| DataAgent_RLBackbone=LLaMA-DCLM, Reweighting Space=Data sources2025.07 | 42.14 | — | |
| G-DesignerAttack Rate=> 50%2026.02 | 41 | 563 | |
| G-DesignerAttack Rate== 50%2026.02 | 40.83 | 566 | |
| DBLBackbone=LLaMA-FWE, Reweighting Space=Data sources2025.07 | 40.7 | — | |
| Mula-1BModel Type=Dense2026.04 | 39.8 | — | |
| DataAgent_RLBackbone=LLaMA-Nemo, Reweighting Space=Data sources2025.07 | 39.8 | — | |
| AgentPruneAttack Rate=> 50%2026.02 | 39.17 | 471 | |
| DataAgent_SFTBackbone=LLaMA-Nemo, Reweighting Space=Data sources2025.07 | 39.04 | — | |
| DataAgent_RLBackbone=LLaMA-FWE, Reweighting Space=Data sources2025.07 | 38.8 | — | |
| DataAgent_SFTBackbone=LLaMA-FWE, Reweighting Space=Data sources2025.07 | 38.28 | — | |
| Base ModelBackbone=LLaMA-FWE, Reweighting Space=Data sources2025.07 | 37.8 | — | |
| Base ModelBackbone=LLaMA-Nemo, Reweighting Space=Data sources2025.07 | 37.6 | — | |
| Complete GraphAttack Rate=> 50%2026.02 | 37.33 | 643 | |
| Llama-3.2-1B DenseConfig=Dense2026.03 | 37.2 | — | |
| Base ModelBackbone=LLaMA-DCLM, Reweighting Space=Data sources2025.07 | 37 | — | |
| RegMixBackbone=LLaMA-DCLM, Reweighting Space=Data sources2025.07 | 36.6 | — | |
| RegMixBackbone=LLaMA-FWE, Reweighting Space=Data sources2025.07 | 36.45 | — | |
| LFT-3BASiLConfig=4:8+64LR, Fine-Tuning=LoRA2026.03 | 36.4 | — | |
| Random GraphAttack Rate=> 50%2026.02 | 36.33 | 550 | |
| LFT-Hassle-free-SparseGPT-TMConfig=4:8+64LR, Fine-Tuning=LoRA2026.03 | 36 | — | |
| RegMixBackbone=LLaMA-Nemo, Reweighting Space=Data sources2025.07 | 35.9 | — | |
| OriginalModel=Mixtral-8x22B, Scope=None2026.04 | 35.8 | — | |
| DEKModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=2e2026.04 | 35.8 | — | |
| LFT-3BASiL-TMConfig=4:8+64LR, Fine-Tuning=LoRA2026.03 | 35.2 | — | |
| Count-guidedModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=2e2026.04 | 35.2 | — | |
| GRAPEModel=Mixtral-8x22B, Scope=Global, Number of experts pruned (e)=2e2026.04 | 35.2 | — | |
| LFT-Hassle-free-SparseGPT-TMConfig=2:4+64LR, Fine-Tuning=LoRA2026.03 | 35 | — | |
| EnumerateModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=2e2026.04 | 35 | — | |
| DBLBackbone=LLaMA-DCLM, Reweighting Space=Data sources2025.07 | 35 | — | |
| LFT-Hassle-free-ALPS-TMConfig=2:4+64LR, Fine-Tuning=LoRA2026.03 | 34.2 | — | |
| DataAgent_SFTBackbone=LLaMA-DCLM, Reweighting Space=Data sources2025.07 | 34.2 | — | |
| LFT-3BASiLConfig=2:4+64LR, Fine-Tuning=LoRA2026.03 | 34 | — | |
| DBLBackbone=LLaMA-Nemo, Reweighting Space=Data sources2025.07 | 34 | — | |
| LFT-Hassle-free-ALPSConfig=4:8+64LR, Fine-Tuning=LoRA2026.03 | 33.8 | — | |
| LFT-Hassle-free-ALPS-TMConfig=4:8+64LR, Fine-Tuning=LoRA2026.03 | 33.8 | — | |
| Count-guidedModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=2e2026.04 | 33.8 | — | |
| LFT-OATS-TMConfig=4:8+64LR, Fine-Tuning=LoRA2026.03 | 33.6 | — | |
| LFT-Hassle-free-ALPSConfig=2:4+64LR, Fine-Tuning=LoRA2026.03 | 33.6 | — | |
| LFT-3BASiL-TMConfig=2:4+64LR, Fine-Tuning=LoRA2026.03 | 33.6 | — | |
| RegMixBackbone=Pythia-1.4B, Reweighting Space=Data sources2025.07 | 33.4 | — | |
| LFT-3BASiL-TMConfig=3:8+64LR, Fine-Tuning=LoRA2026.03 | 33.2 | — | |
| LFT-Hassle-free-SparseGPTConfig=4:8+64LR, Fine-Tuning=LoRA2026.03 | 33.2 | — | |
| DEKModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=4e2026.04 | 33.2 | — | |
| Router-guidedModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=2e2026.04 | 33.2 | — | |
| Base ModelBackbone=Pythia-1.4B, Reweighting Space=Data sources2025.07 | 33.2 | — | |
| LFT-OATS-TMConfig=2:4+64LR, Fine-Tuning=LoRA2026.03 | 33 | — | |
| OriginalModel=DeepSeek-MoE, Scope=None2026.04 | 32.8 | — | |
| Count-guidedModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=4e2026.04 | 32.4 | — | |
| DEKModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=2e2026.04 | 32.2 | — | |
| DEKModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=4e2026.04 | 32.2 | — | |
| LFT-Hassle-free-ALPSConfig=3:8+64LR, Fine-Tuning=LoRA2026.03 | 32 | — | |
| GRAPEModel=Mixtral-8x22B, Scope=Global, Number of experts pruned (e)=4e2026.04 | 32 | — | |
| GRAPEModel=DeepSeek-MoE, Scope=Global, Number of experts pruned (e)=2e2026.04 | 32 | — | |
| LFT-3BASiLConfig=3:8+64LR, Fine-Tuning=LoRA2026.03 | 31.8 | — | |
| LFT-OATSConfig=2:4+64LR, Fine-Tuning=LoRA2026.03 | 31.6 | — | |
| LFT-Hassle-free-SparseGPTConfig=2:4+64LR, Fine-Tuning=LoRA2026.03 | 31.4 | — | |
| Router-guidedModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=4e2026.04 | 31.4 | — | |
| LFT-Hassle-free-ALPS-TMConfig=3:8+64LR, Fine-Tuning=LoRA2026.03 | 31.2 | — | |
| GRAPEModel=DeepSeek-MoE, Scope=Global, Number of experts pruned (e)=4e2026.04 | 31.2 | — | |
| EnumerateModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=4e2026.04 | 31.1 | — | |
| LFT-Hassle-free-SparseGPTConfig=3:8+64LR, Fine-Tuning=LoRA2026.03 | 30.6 | — | |
| DataAgent_RLBackbone=Pythia-1.4B, Reweighting Space=Data sources2025.07 | 30.6 | — | |
| LFT-Hassle-free-SparseGPT-TMConfig=3:8+64LR, Fine-Tuning=LoRA2026.03 | 30.2 | — |