Science Question Answering on ARC Challenge (Accuracy, AVG, Delta)
58.1AccuracyQwen 8B (w/ LoopUS)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Qwen 8B (w/ LoopUS)Model=Qwen 8B, Setting=w/ LoopUS, Evaluation protocol=zero-shot2026.05 | 58.1 | 65.4 | 2.2 | |
| Phi-4 14B (w/ LoopUS)Model=Phi-4 14B, Setting=w/ LoopUS, Evaluation protocol=zero-shot2026.05 | 57.7 | 68.6 | 1.7 | |
| Phi-4 14B (w/o LoopUS)Model=Phi-4 14B, Setting=w/o LoopUS, Evaluation protocol=zero-shot2026.05 | 55.8 | 67 | — | |
| Qwen 8B (w/o LoopUS)Model=Qwen 8B, Setting=w/o LoopUS, Evaluation protocol=zero-shot2026.05 | 55.4 | 63.2 | — | |
| Qwen 4B (w/ LoopUS)Model=Qwen 4B, Setting=w/ LoopUS, Evaluation protocol=zero-shot2026.05 | 54 | 62.1 | 1.8 | |
| Qwen 4B (w/o LoopUS)Model=Qwen 4B, Setting=w/o LoopUS, Evaluation protocol=zero-shot2026.05 | 50.4 | 60.3 | — | |
| Qwen 1.7B (w/ LoopUS)Model=Qwen 1.7B, Setting=w/ LoopUS, Evaluation protocol=zero-shot2026.05 | 43.1 | 55.3 | 1.6 | |
| Qwen 1.7B (w/o LoopUS)Model=Qwen 1.7B, Setting=w/o LoopUS, Evaluation protocol=zero-shot2026.05 | 40.2 | 53.7 | — |