Commonsense Reasoning on CommonsenseQA (Accuracy)
64.2AccuracyLightReasoner
Evaluation Results
| Method | Links | |
|---|---|---|
| LightReasoner2025.10 | 64.2 | |
| Baseline2025.10 | 62.6 | |
| NITPModel scale=9bA1b, Evaluation=few-shot, Context length=81922026.05 | 49.96 | |
| Full-data Fine-tuningBackbone=LLAMA-3.2-1B, Coreset size (%)=100%, Fine-tuning protocol=Full, Evaluation protocol=EleutherAI lm-evaluation-harness2025.10 | 48.24 | |
| NTPModel scale=9bA1b, Evaluation=few-shot, Context length=81922026.05 | 45.7 | |
| TRIMBackbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.10 | 40.76 | |
| LESSBackbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.10 | 39.1 | |
| BM25Backbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.10 | 38.88 | |
| NITPModel scale=3bA0.5b, Evaluation=few-shot, Context length=81922026.05 | 37.92 | |
| DSIRBackbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.10 | 37.16 | |
| RDSBackbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.10 | 36.16 | |
| TAGCOSBackbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.10 | 34.72 | |
| NTPModel scale=3bA0.5b, Evaluation=few-shot, Context length=81922026.05 | 34.15 | |
| S2LBackbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.10 | 34.1 | |
| RandomBackbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.10 | 34.05 | |
| CLDBackbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.10 | 33.1 | |
| Pretrained (no Fine-tuning)Backbone=LLAMA-3.2-1B, Fine-tuning protocol=None, Evaluation protocol=EleutherAI lm-evaluation-harness2025.10 | 29.32 | |
| NITPModel scale=1.9bA0.3b, Evaluation=few-shot, Context length=81922026.05 | 26.61 | |
| NTPModel scale=1.9bA0.3b, Evaluation=few-shot, Context length=81922026.05 | 25.38 |