Commonsense Reasoning on ARC-c (zero-shot)
40.7Accuracy (zero-shot)Qwen3-8B + CAT-Q
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen3-8B + CAT-Q#Tokens=1M, Quantization=W1.58A162026.06 | 40.7 | |
| Qwen3-4B + CAT-Q#Tokens=2M, Quantization=W1.58A162026.06 | 39.08 | |
| Qwen3-4B + CAT-Q#Tokens=1M, Quantization=W1.58A162026.06 | 36.95 | |
| TernaryLLM-8B+KD#Tokens=1T, Quantization=W1.58A162026.06 | 36.4 | |
| TequilaLLM-3B#Tokens=10B, Quantization=W1.58A162026.06 | 34.6 | |
| TriLM-3.9B#Tokens=300B, Quantization=W1.58A162026.06 | 31.9 | |
| BitNetV1-7B#Tokens=100B, Quantization=W1.58A162026.06 | 31.74 | |
| BitNetV1-3.9B#Tokens=100B, Quantization=W1.58A162026.06 | 28.7 | |
| Qwen3-1.7B + CAT-Q#Tokens=1M, Quantization=W1.58A162026.06 | 28.16 | |
| TriLM-1.5B#Tokens=300B, Quantization=W1.58A162026.06 | 24.7 | |
| BitNetV1-1.3B#Tokens=100B, Quantization=W1.58A162026.06 | 24.2 |