Science Question Answering on ARC-Challenge 25-shot
0.637AccuracyQwen2.5-7B
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen2.5-7BModel Type=Dense, Active Parameters=7B, Total Parameters=7B, Shots=252026.02 | 0.637 | |
| Qwen2.5-3BModel Type=Dense, Active Parameters=3B, Total Parameters=3B, Shots=252026.02 | 0.565 | |
| ExpertWeaver-E64-A14-S2(Qwen2.5-7B)Model Type=MoE, Active Parameters=3.5B, Total Parameters=7B, Training Budget=200B tokens, Shots=252026.02 | 0.563 | |
| OLMoE-1B-7BModel Type=MoE, Active Parameters=1B, Total Parameters=7B, Shots=252026.02 | 0.556 | |
| Qwen2.5-1.5BModel Type=Dense, Active Parameters=1.5B, Total Parameters=1.5B, Shots=252026.02 | 0.549 | |
| SmolLM2-1.7BModel Type=Dense, Active Parameters=1.7B, Total Parameters=1.7B, Shots=252026.02 | 0.532 | |
| Gemma-2-2bModel Type=Dense, Active Parameters=2B, Total Parameters=2B, Shots=252026.02 | 0.526 | |
| Llama-3.2-3BModel Type=Dense, Active Parameters=3B, Total Parameters=3B, Shots=252026.02 | 0.505 | |
| OLMo-7BModel Type=Dense, Active Parameters=7B, Total Parameters=7B, Shots=252026.02 | 0.451 | |
| LLaMA-MoE-v1-3.5BModel Type=MoE, Active Parameters=3.5B, Shots=252026.02 | 0.442 | |
| OLMoE-1B-7B*Model Type=MoE, Active Parameters=1B, Total Parameters=7B, Training Budget=500B tokens, Shots=252026.02 | 0.439 | |
| Sheared-LLaMA-2.7BModel Type=Dense, Active Parameters=2.7B, Total Parameters=2.7B, Shots=252026.02 | 0.416 | |
| INCITE-Base-3BModel Type=Dense, Active Parameters=3B, Total Parameters=3B, Shots=252026.02 | 0.403 | |
| LLaMA-MoE-v2-3.5BModel Type=MoE, Active Parameters=3.5B, Shots=252026.02 | 0.402 | |
| Open-LLaMA-3B-v2Model Type=Dense, Active Parameters=3B, Total Parameters=3B, Shots=252026.02 | 0.401 | |
| ExpertWeaver-E64-A14-S2(OLMo-7B)Model Type=MoE, Active Parameters=1B, Total Parameters=7B, Training Budget=200B tokens, Shots=252026.02 | 0.388 | |
| Pythia-2.8BModel Type=Dense, Active Parameters=2.8B, Total Parameters=2.8B, Shots=252026.02 | 0.367 | |
| OPT-2.7BModel Type=Dense, Active Parameters=2.7B, Total Parameters=2.7B, Shots=252026.02 | 0.34 | |
| OpenMoE-3B-9BModel Type=MoE, Active Parameters=3B, Total Parameters=9B, Shots=252026.02 | 0.333 |