List your company
ResearchFormerly WizWand

Zero-shot Classification on ARC-Challenge, ARC-Easy, PIQA, and WinoGrande

73.9AccuracyFull-precision
37.18846.71956.2565.781Mar 30, 2026
Updated 6mo ago

Evaluation results

MethodLinks
Full-precisionModel=Qwen3-14B, Quantization setting=FP2026.0373.9
JointQModel=Qwen3-14B, Quantization setting=4b/1282026.0373.9
GPTQModel=Qwen3-14B, Quantization setting=4b/1282026.0373.6
JointQModel=Qwen3-14B, Quantization setting=4b/ch2026.0372.3
GPTQModel=Qwen3-14B, Quantization setting=4b/ch2026.0372
Full-precisionModel=Llama-3-8B, Quantization setting=FP2026.0371.5
Full-precisionModel=Qwen3-8B, Quantization setting=FP2026.0370.7
JointQModel=Llama-3-8B, Quantization setting=4b/1282026.0370.5
JointQModel=Qwen3-14B, Quantization setting=3b/1282026.0370.1
GPTQModel=Qwen3-14B, Quantization setting=3b/1282026.0369.8
GPTQModel=Llama-3-8B, Quantization setting=4b/1282026.0369.7
GPTQModel=Qwen3-8B, Quantization setting=4b/1282026.0369.7
JointQModel=Qwen3-8B, Quantization setting=4b/1282026.0369.5
JointQModel=Llama-3-8B, Quantization setting=4b/ch2026.0367.6
GPTQModel=Qwen3-8B, Quantization setting=4b/ch2026.0367.2
JointQModel=Qwen3-8B, Quantization setting=3b/1282026.0366.8
JointQModel=Qwen3-8B, Quantization setting=4b/ch2026.0366
GPTQModel=Qwen3-8B, Quantization setting=3b/1282026.0365.4
JointQModel=Qwen3-14B, Quantization setting=3b/ch2026.0365.1
JointQModel=Llama-3-8B, Quantization setting=3b/1282026.0365
JointQModel=Llama-3-8B, Quantization setting=3b/ch2026.0358.3
GPTQModel=Qwen3-14B, Quantization setting=3b/ch2026.0358.1
GPTQModel=Llama-3-8B, Quantization setting=4b/ch2026.0353.3
GPTQModel=Llama-3-8B, Quantization setting=3b/1282026.0352.1
JointQModel=Qwen3-8B, Quantization setting=3b/ch2026.0350.6
GPTQModel=Qwen3-8B, Quantization setting=3b/ch2026.0349.6
GPTQModel=Llama-3-8B, Quantization setting=3b/ch2026.0338.6