Question Answering on ARC Challenge 25-shot (test)
69.8AccuracyExpertWeaver-Instruct
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ExpertWeaver-InstructInstruction-tuned=true2026.02 | 69.8 | — | — | |
| Uniform Regularized LoopingModel=Gemma, Size=9B, Shots=25-shot2026.02 | 68.09 | — | — | |
| Moving Average Regularized LoopingModel=Gemma, Size=9B, Shots=25-shot2026.02 | 68.09 | — | — | |
| BaselineModel=Gemma, Size=9B, Shots=25-shot2026.02 | 67.75 | — | — | |
| Auto-Align Regularized LoopingModel=Gemma, Size=9B, Shots=25-shot2026.02 | 67.58 | — | — | |
| Noise AblationModel=Gemma, Size=9B, Shots=25-shot2026.02 | 67.49 | — | — | |
| Auto-Align Regularized LoopingModel=Llama, Size=8B, Shots=25-shot2026.02 | 60.07 | — | — | |
| BaselineModel=Llama, Size=8B, Shots=25-shot2026.02 | 59.73 | — | — | |
| Moving Average Regularized LoopingModel=Llama, Size=8B, Shots=25-shot2026.02 | 59.73 | — | — | |
| Uniform Regularized LoopingModel=Llama, Size=8B, Shots=25-shot2026.02 | 59.64 | — | — | |
| Noise AblationModel=Llama, Size=8B, Shots=25-shot2026.02 | 59.64 | — | — | |
| OLMoE-1B-7B2026.02 | 55.63 | — | — | |
| Uniform Regularized LoopingModel=Gemma, Size=2B, Shots=25-shot2026.02 | 53.41 | — | — | |
| Auto-Align Regularized LoopingModel=Gemma, Size=2B, Shots=25-shot2026.02 | 52.99 | — | — | |
| Moving Average Regularized LoopingModel=Gemma, Size=2B, Shots=25-shot2026.02 | 52.9 | — | — | |
| BaselineModel=Gemma, Size=2B, Shots=25-shot2026.02 | 52.82 | — | — | |
| Noise AblationModel=Gemma, Size=2B, Shots=25-shot2026.02 | 52.65 | — | — | |
| LLaMA-MoE-3B-7B2026.02 | 44.03 | — | — | |
| LLaMA-MoE-v22026.02 | 40.2 | — | — | |
| HATifiedshots=25-shot, evaluation_mode=Pre-training2026.03 | — | 53.7 | 6.57 | |
| Llamashots=25-shot, evaluation_mode=Pre-training2026.03 | — | 47.4 | 3.68 | |
| T-Freeshots=25-shot, evaluation_mode=Pre-training2026.03 | — | 59.1 | 6.57 |