List your company
ResearchFormerly WizWand

Commonsense Reasoning on CommonsenseQA (Accuracy)

64.2AccuracyLightReasoner
23.827234.308644.7955.2714Oct 8, 2025Nov 15, 2025Dec 23, 2025Jan 30, 2026Mar 9, 2026Apr 16, 2026May 24, 2026
Updated 5mo ago

Evaluation results

MethodLinks
LightReasoner2025.1064.2
Baseline2025.1062.6
NITPModel scale=9bA1b, Evaluation=few-shot, Context length=81922026.0549.96
Full-data Fine-tuningBackbone=LLAMA-3.2-1B, Coreset size (%)=100%, Fine-tuning protocol=Full, Evaluation protocol=EleutherAI lm-evaluation-harness2025.1048.24
NTPModel scale=9bA1b, Evaluation=few-shot, Context length=81922026.0545.7
TRIMBackbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.1040.76
LESSBackbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.1039.1
BM25Backbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.1038.88
NITPModel scale=3bA0.5b, Evaluation=few-shot, Context length=81922026.0537.92
DSIRBackbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.1037.16
RDSBackbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.1036.16
TAGCOSBackbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.1034.72
NTPModel scale=3bA0.5b, Evaluation=few-shot, Context length=81922026.0534.15
S2LBackbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.1034.1
RandomBackbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.1034.05
CLDBackbone=LLAMA-3.2-1B, Coreset size (%)=5%, Fine-tuning protocol=Coreset fine-tuning, Evaluation protocol=EleutherAI lm-evaluation-harness2025.1033.1
Pretrained (no Fine-tuning)Backbone=LLAMA-3.2-1B, Fine-tuning protocol=None, Evaluation protocol=EleutherAI lm-evaluation-harness2025.1029.32
NITPModel scale=1.9bA0.3b, Evaluation=few-shot, Context length=81922026.0526.61
NTPModel scale=1.9bA0.3b, Evaluation=few-shot, Context length=81922026.0525.38