Question Answering Task on ARC-C (test)
92.49AccuracyBLoB
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BLoBParams (M)=5.4032026.06 | 92.49 | 4.46 | 0.28 | |
| FVAE-LoRAParams (M)=33.532026.06 | 91.93 | 4.52 | 0.27 | |
| BaRAParams (M)=5.6082026.06 | 91.89 | 3.75 | 0.25 | |
| C-LoRAParams (M)=4.2752026.06 | 91.74 | 4.44 | 0.28 | |
| ScalaBLParams (M)=3.7692026.06 | 91.06 | 4.41 | 0.27 | |
| MLEParams (M)=3.7682026.06 | 90.88 | 7.72 | 0.44 | |
| EnsembleParams (M)=11.3052026.06 | 90.84 | 6.18 | 0.33 | |
| LaplaceParams (M)=3.7682026.06 | 90.51 | 4.75 | 0.32 | |
| MC-DropoutParams (M)=3.7682026.06 | 90.37 | 7.63 | 0.43 | |
| MAPParams (M)=3.7682026.06 | 90.2 | 7.89 | 0.46 | |
| LLaDA 8BPost-train=SFT, Model Type=Diffusion, Number of shots=0, Evaluation Protocol=same protocol (*)2026.01 | 88.5 | — | — | |
| LLaMA3 8BPost-train=SFT+RL, Model Type=Autoregressive, Number of shots=0, Evaluation Protocol=same protocol (*)2026.01 | 82.4 | — | — | |
| VDLMPost-train=SFT+RL, Model Type=Diffusion, Number of shots=0, Evaluation Protocol=same protocol (*)2026.01 | 80.8 | — | — | |
| BaselineModel=Mixtral-8x7B-Instruct, Capacity Factor=1.52025.03 | 71.7 | — | — | |
| Expanded DropModel=Mixtral-8x7B-Instruct, Capacity Factor=1.52025.03 | 71.5 | — | — | |
| Token DropModel=Mixtral-8x7B-Instruct, Capacity Factor=1.52025.03 | 71.2 | — | — | |
| Expert DropModel=Mixtral-8x7B-Instruct, Capacity Factor=1.52025.03 | 67.1 | — | — | |
| BaselineModel=DeepSeek-V2-Lite-Chat, Capacity Factor=2.02025.03 | 61 | — | — | |
| Expanded DropModel=DeepSeek-V2-Lite-Chat, Capacity Factor=2.02025.03 | 60.4 | — | — | |
| Token DropModel=DeepSeek-V2-Lite-Chat, Capacity Factor=2.02025.03 | 59.3 | — | — | |
| Token DropModel=OLMoE-Instruct, Capacity Factor=2.02025.03 | 57.2 | — | — | |
| Expanded DropModel=OLMoE-Instruct, Capacity Factor=2.02025.03 | 57.1 | — | — | |
| Expert DropModel=DeepSeek-V2-Lite-Chat, Capacity Factor=2.02025.03 | 57.1 | — | — | |
| BaselineModel=OLMoE-Instruct, Capacity Factor=2.02025.03 | 57 | — | — | |
| Expert DropModel=OLMoE-Instruct, Capacity Factor=2.02025.03 | 54.4 | — | — | |
| BaselineModel=Qwen1.5-MoE-Chat, Capacity Factor=1.52025.03 | 54.1 | — | — | |
| Expanded DropModel=Qwen1.5-MoE-Chat, Capacity Factor=1.52025.03 | 53.4 | — | — | |
| Token DropModel=Qwen1.5-MoE-Chat, Capacity Factor=1.52025.03 | 53 | — | — | |
| Expert DropModel=Qwen1.5-MoE-Chat, Capacity Factor=1.52025.03 | 52.9 | — | — | |
| Deepseek 7BPost-train=SFT+RL, Model Type=Autoregressive, Number of shots=0, Evaluation Protocol=RL alignment (†)2026.01 | 49.4 | — | — |