List your company
ResearchFormerly WizWand

Common-sense Reasoning on ARC-C OOD Small Shift

81.6AccuracyC-LoRA
78.563279.351680.1480.9284Apr 3, 2026Apr 18, 2026May 3, 2026May 18, 2026Jun 2, 2026Jun 17, 2026Jul 2, 2026
Updated 4mo ago

Evaluation results

MethodLinks
C-LoRABackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C, Monte Carlo samples (M)=102026.0781.67.060.55
DALorRABackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C, Monte Carlo samples (M)=102026.0781.66.230.54
MLEBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C2026.0781.4814.451.07
MCDBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C, Monte Carlo samples (M)=102026.0781.4213.631.03
TFBBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C, Monte Carlo samples (M)=102026.0781.127.120.6
PoLAR-VBLL (w/o LA)Backbone=Llama-3.1-8B, Training dataset=OBQA2026.0481.117.650.56
PoLAR-VBLLBackbone=Llama-3.1-8B, Training dataset=OBQA2026.0481.114.550.53
PoLAR-VBLLBackbone=Llama-3.1-8B, Fine-tuning Steps=5000, Training Dataset=OBQA2026.0481.114.550.53
LABackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C2026.0781.087.831.17
PoLAR-MLEBackbone=Llama-3.1-8B, Training dataset=OBQA2026.0480.9710.840.68
PoLAR-LABackbone=Llama-3.1-8B, Training dataset=OBQA2026.0480.977.090.56
PoLAR-LA-LLBackbone=Llama-3.1-8B, Training dataset=OBQA2026.0480.979.450.57
TFB-LLBackbone=Llama-3.1-8B, Training dataset=OBQA2026.0480.977.860.58
TFBBackbone=Llama-3.1-8B, Training dataset=OBQA2026.0480.95.10.54
PoLAR-BLoBBackbone=Llama-3.1-8B, Training dataset=OBQA2026.0480.346.520.55
C-LoRABackbone=Llama-3.1-8B, Training dataset=OBQA2026.0480.0713.940.83
MAPBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C2026.0779.9816.461.19
ENSBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C2026.0779.6211.370.72
MLEBackbone=Llama-3.1-8B, Training dataset=OBQA2026.0479.3313.890.9
LABackbone=Llama-3.1-8B, Training dataset=OBQA2026.0479.335.850.54
MLEBackbone=Llama-3.1-8B, Fine-tuning Steps=5000, Training Dataset=OBQA2026.0479.3313.890.9
BLoBBackbone=Llama-3.1-8B, Training dataset=OBQA2026.0479.14.870.55
BLoBBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C, Monte Carlo samples (M)=102026.0778.687.020.61