Question Answering on ARC Challenge (Accuracy and PEEM Scores)
93.7AccuracyGPT-4o-mini
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-4o-miniTask Model=GPT-4o-mini2026.03 | 93.7 | 4.815 | 4.81 | 4.702 | |
| CascadeDebateModel Family=Llama-3.2, Base Model Size=1B, Large Model Size=3B2026.04 | 92.89 | — | — | — | |
| Gemini-2.5-FlashTask Model=Gemini-2.5-Flash2026.03 | 91.4 | 4.721 | 4.782 | 4.595 | |
| Qwen-2.5-7B-ITTask Model=Qwen-2.5-7B-IT2026.03 | 89.8 | 4.702 | 4.692 | 4.555 | |
| Standard CascadeModel Family=Qwen2.5, Base Model Size=1.5B, Large Model Size=3B2026.04 | 89.8 | — | — | — | |
| CascadeDebateModel Family=Qwen2.5, Base Model Size=1.5B, Large Model Size=3B2026.04 | 85.78 | — | — | — | |
| S_multi(M_large)Model Family=Qwen2.5, Base Model Size=1.5B, Large Model Size=3B2026.04 | 84.44 | — | — | — | |
| Standard CascadeModel Family=Llama-3.2, Base Model Size=1B, Large Model Size=3B2026.04 | 84.3 | — | — | — | |
| LLaMA-3.1-8B-ITTask Model=LLaMA-3.1-8B-IT2026.03 | 83.4 | 4.461 | 4.577 | 4.35 | |
| S_single(M_large)Model Family=Qwen2.5, Base Model Size=1.5B, Large Model Size=3B2026.04 | 82.78 | — | — | — | |
| S_multi(M_large)Model Family=Llama-3.2, Base Model Size=1B, Large Model Size=3B2026.04 | 81.67 | — | — | — | |
| S_single(M_large)Model Family=Llama-3.2, Base Model Size=1B, Large Model Size=3B2026.04 | 78.33 | — | — | — | |
| S_multi(M_base)Model Family=Qwen2.5, Base Model Size=1.5B, Large Model Size=3B2026.04 | 74.67 | — | — | — | |
| S_single(M_base)Model Family=Qwen2.5, Base Model Size=1.5B, Large Model Size=3B2026.04 | 71.89 | — | — | — | |
| Gemma-2-9B-ITTask Model=Gemma-2-9B-IT2026.03 | 67.9 | 4.168 | 4.246 | 4.105 | |
| UnprunedBackbone=Llama-3.1-8B2026.04 | 56 | — | — | — | |
| S_multi(M_base)Model Family=Llama-3.2, Base Model Size=1B, Large Model Size=3B2026.04 | 54.78 | — | — | — | |
| S_single(M_base)Model Family=Llama-3.2, Base Model Size=1B, Large Model Size=3B2026.04 | 50.67 | — | — | — | |
| SmolLM-1.7B-InstructShots=5-shot2026.03 | 43.9 | — | — | — | |
| DATEDGPT-BASE-2024Shots=5-shot2026.03 | 40.2 | — | — | — | |
| DATEDGPT-BASE-2023Shots=5-shot2026.03 | 39.8 | — | — | — | |
| DATEDGPT-BASE-2017Shots=5-shot2026.03 | 39.3 | — | — | — | |
| DATEDGPT-BASE-2021Shots=5-shot2026.03 | 38.4 | — | — | — | |
| DATEDGPT-BASE-2018Shots=5-shot2026.03 | 38.3 | — | — | — | |
| DATEDGPT-BASE-2016Shots=5-shot2026.03 | 37.8 | — | — | — | |
| DATEDGPT-BASE-2020Shots=5-shot2026.03 | 37.8 | — | — | — | |
| DATEDGPT-BASE-2014Shots=5-shot2026.03 | 37 | — | — | — | |
| DATEDGPT-BASE-2022Shots=5-shot2026.03 | 36.8 | — | — | — | |
| DATEDGPT-BASE-2015Shots=5-shot2026.03 | 36 | — | — | — | |
| TinyLlama-1.1BShots=5-shot2026.03 | 35.5 | — | — | — | |
| DATEDGPT-BASE-2019Shots=5-shot2026.03 | 35 | — | — | — | |
| DATEDGPT-BASE-2013Shots=5-shot2026.03 | 34.6 | — | — | — | |
| GPT2-XLShots=5-shot2026.03 | 29.9 | — | — | — | |
| OPT-1.3BShots=5-shot2026.03 | 29.2 | — | — | — | |
| Pythia-1BShots=5-shot2026.03 | 28.2 | — | — | — | |
| OWL (channel)Backbone=Llama-3.1-8B2026.04 | 27 | — | — | — | |
| SCAR-LPBackbone=Llama-3.1-8B2026.04 | 26 | — | — | — | |
| Wanda (channel)Backbone=Llama-3.1-8B2026.04 | 25 | — | — | — | |
| RandomBackbone=Llama-3.1-8B2026.04 | 24 | — | — | — | |
| SparseGPT (channel)Backbone=Llama-3.1-8B2026.04 | 24 | — | — | — | |
| SCAR-ConnBackbone=Llama-3.1-8B2026.04 | 24 | — | — | — | |
| SlimLLM (channel)Backbone=Llama-3.1-8B2026.04 | 23 | — | — | — | |
| Act. L2Backbone=Llama-3.1-8B2026.04 | 23 | — | — | — | |
| SCAR-ProtBackbone=Llama-3.1-8B2026.04 | 23 | — | — | — | |
| Magnitude (channel)Backbone=Llama-3.1-8B2026.04 | 22 | — | — | — | |
| RIA (channel)Backbone=Llama-3.1-8B2026.04 | 22 | — | — | — | |
| LLM-Pruner (channel)Backbone=Llama-3.1-8B2026.04 | 21 | — | — | — | |
| RQBackbone=Llama-3.1-8B2026.04 | 21 | — | — | — | |
| Gaussian MI (analytic)Backbone=Llama-3.1-8B2026.04 | 20 | — | — | — | |
| FLAP (channel)Backbone=Llama-3.1-8B2026.04 | 19 | — | — | — |