List your company
ResearchFormerly WizWand

Science Question Answering on ARC Challenge (Accuracy)

66.1Accuracy (ARC)No pruning
23.35634.45345.5556.647Mar 17, 2026Mar 18, 2026Mar 19, 2026Mar 21, 2026Mar 22, 2026Mar 23, 2026Mar 25, 2026
Updated 7mo ago

Evaluation results

MethodLinks
No pruningSparsity=0, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.0366.1
Fanar-27BSize=27B, Few-shot settings=0-shot2026.0365.61
Fanar-1-9b-instructSize=9B, Few-shot settings=0-shot2026.0365.19
Llama-3.3-70b-InstructSize=70B, Few-shot settings=0-shot2026.0363.05
Qwen3-32BSize=32B, Few-shot settings=0-shot2026.0360.84
AceGPT-v2-70B-ChatSize=70B, Few-shot settings=0-shot2026.0360.07
Gemma-3-27B-itSize=27B, Few-shot settings=0-shot2026.0359.98
Jais-2-70B-ChatSize=70B, Few-shot settings=0-shot2026.0359.3
Allam-7B-Instruct-preview-v2Size=7B, Few-shot settings=0-shot2026.0358.62
AceGPT-v2-32B-ChatSize=32B, Few-shot settings=0-shot2026.0353.92
Magnitude-DimSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.0350.6
DIETSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.0349.6
KarnakSize=40B, Few-shot settings=0-shot2026.0347.35
PuDDingSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.0332.1
DIETSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.0332
Magnitude-DimSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.0330.8
PuDDingSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.0330.3
SliceGPTSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.0325
SliceGPTSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.0325