List your company
ResearchFormerly WizWand

Question Answering on ARC (accuracy)

94.2AccuracyGPT-5 nano
20.900839.930458.9677.9896May 8, 2026May 12, 2026May 16, 2026May 21, 2026May 25, 2026May 29, 2026Jun 3, 2026
Updated 5mo ago

Evaluation results

MethodLinks
GPT-5 nanoScale=nano2026.0594.2
Qwen3-8BParameters=8B2026.0593
Ministral-3-8BParameters=8B2026.0592.1
gpt-oss-20bParameters=20B2026.0592.1
Qwen3-4BParameters=4B2026.0590.3
gemma-3-12b-itParameters=12B2026.0589.9
FPBackbone=LLADA-1.5-8B, Weight Bits=Full Precision, Activation Bits=Full Precision2026.0688.5
STaR-QuantBackbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.0685.12
QuaRotBackbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.0684.23
DLLMQuant++Backbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.0684.18
DLLMQuant+Backbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.0683.38
AWQBackbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.0682.22
RTNBackbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.0681.77
EngGPT2-16B-A3BParameters=16B2026.0580.2
Llama-3.1-8BParameters=8B2026.0578.1
gemma-3-4b-itParameters=4B2026.0575.3
Moonlight-16B-A3BParameters=16B2026.0574.1
Velvet-14BParameters=14B2026.0570.3
Llama-3.2-3BParameters=3B2026.0569.8
LLaMAntino-3-8BParameters=8B2026.0565.6
FPBackbone=DREAM-7B, Weight Bits=Full Precision, Activation Bits=Full Precision2026.0659.8
STaR-QuantBackbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.0659.76
DLLMQuant++Backbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.0659.43
DLLMQuant+Backbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.0658.38
LoopQModel=Ouro 2.6B, Quantization=W4A82026.0558.19
QuaRotBackbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.0658.18
AWQBackbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.0657.82
RTNBackbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.0654.34
FastwebMIIA-7BParameters=7B2026.0553.5
LoopQModel=Ouro 1.4B, Quantization=W4A82026.0551.45
LoopQModel=Ouro 2.6B, Quantization=W4A42026.0550.34
LoopQModel=Ouro 1.4B, Quantization=W4A42026.0548.81
deepseek-moe-16bParameters=16B2026.0547
STaR-QuantBackbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.0644.23
QuaRotBackbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.0644.2
DLLMQuant++Backbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.0644.18
FPBackbone=LLADA-8B, Weight Bits=Full Precision, Activation Bits=Full Precision2026.0644.03
DLLMQuant+Backbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.0643.44
AWQBackbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.0642.92
RTNBackbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.0641.83
LoopQModel=Parcae 370M, Quantization=W4A42026.0529.61
LoopQModel=Parcae 370M, Quantization=W4A82026.0529.29
Minerva-7BParameters=7B2026.0525.9
LoopQModel=LoopFormer 3×8, Quantization=W4A42026.0525.61
LoopQModel=LoopFormer 3×8, Quantization=W4A82026.0523.72