Question Answering on ARC (accuracy)
94.2AccuracyGPT-5 nano
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-5 nanoScale=nano2026.05 | 94.2 | |
| Qwen3-8BParameters=8B2026.05 | 93 | |
| Ministral-3-8BParameters=8B2026.05 | 92.1 | |
| gpt-oss-20bParameters=20B2026.05 | 92.1 | |
| Qwen3-4BParameters=4B2026.05 | 90.3 | |
| gemma-3-12b-itParameters=12B2026.05 | 89.9 | |
| FPBackbone=LLADA-1.5-8B, Weight Bits=Full Precision, Activation Bits=Full Precision2026.06 | 88.5 | |
| STaR-QuantBackbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.06 | 85.12 | |
| QuaRotBackbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.06 | 84.23 | |
| DLLMQuant++Backbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.06 | 84.18 | |
| DLLMQuant+Backbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.06 | 83.38 | |
| AWQBackbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.06 | 82.22 | |
| RTNBackbone=LLADA-1.5-8B, Weight Bits=4, Activation Bits=42026.06 | 81.77 | |
| EngGPT2-16B-A3BParameters=16B2026.05 | 80.2 | |
| Llama-3.1-8BParameters=8B2026.05 | 78.1 | |
| gemma-3-4b-itParameters=4B2026.05 | 75.3 | |
| Moonlight-16B-A3BParameters=16B2026.05 | 74.1 | |
| Velvet-14BParameters=14B2026.05 | 70.3 | |
| Llama-3.2-3BParameters=3B2026.05 | 69.8 | |
| LLaMAntino-3-8BParameters=8B2026.05 | 65.6 | |
| FPBackbone=DREAM-7B, Weight Bits=Full Precision, Activation Bits=Full Precision2026.06 | 59.8 | |
| STaR-QuantBackbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.06 | 59.76 | |
| DLLMQuant++Backbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.06 | 59.43 | |
| DLLMQuant+Backbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.06 | 58.38 | |
| LoopQModel=Ouro 2.6B, Quantization=W4A82026.05 | 58.19 | |
| QuaRotBackbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.06 | 58.18 | |
| AWQBackbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.06 | 57.82 | |
| RTNBackbone=DREAM-7B, Weight Bits=4, Activation Bits=42026.06 | 54.34 | |
| FastwebMIIA-7BParameters=7B2026.05 | 53.5 | |
| LoopQModel=Ouro 1.4B, Quantization=W4A82026.05 | 51.45 | |
| LoopQModel=Ouro 2.6B, Quantization=W4A42026.05 | 50.34 | |
| LoopQModel=Ouro 1.4B, Quantization=W4A42026.05 | 48.81 | |
| deepseek-moe-16bParameters=16B2026.05 | 47 | |
| STaR-QuantBackbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.06 | 44.23 | |
| QuaRotBackbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.06 | 44.2 | |
| DLLMQuant++Backbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.06 | 44.18 | |
| FPBackbone=LLADA-8B, Weight Bits=Full Precision, Activation Bits=Full Precision2026.06 | 44.03 | |
| DLLMQuant+Backbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.06 | 43.44 | |
| AWQBackbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.06 | 42.92 | |
| RTNBackbone=LLADA-8B, Weight Bits=4, Activation Bits=42026.06 | 41.83 | |
| LoopQModel=Parcae 370M, Quantization=W4A42026.05 | 29.61 | |
| LoopQModel=Parcae 370M, Quantization=W4A82026.05 | 29.29 | |
| Minerva-7BParameters=7B2026.05 | 25.9 | |
| LoopQModel=LoopFormer 3×8, Quantization=W4A42026.05 | 25.61 | |
| LoopQModel=LoopFormer 3×8, Quantization=W4A82026.05 | 23.72 |