Question Answering on ARC (Easy and Challenge Sets)
42.83Accuracy (ARC-C)FP16
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FP16Model=Qwen3-1.7B2025.11 | 42.83 | 69.61 | |
| Quant-OnlyModel=Qwen3-1.7B2025.11 | 37.28 | 61.32 | |
| IntAttentionModel=Qwen3-1.7B2025.11 | 37.03 | 61.87 | |
| IntAttentionModel=Llama-3.2-1B2025.11 | 36.43 | 60.48 | |
| FP16Model=Llama-3.2-1B2025.11 | 36.18 | 60.48 | |
| Quant-OnlyModel=Llama-3.2-1B2025.11 | 35.84 | 60.56 | |
| Quant-OnlyModel=OPT-1.3B2025.11 | 29.69 | 50.97 | |
| FP16Model=OPT-1.3B2025.11 | 29.61 | 50.97 | |
| IntAttentionModel=OPT-1.3B2025.11 | 29.18 | 50.8 |