Common-sense Reasoning on ARC-C OOD Small Shift
81.6AccuracyC-LoRA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| C-LoRABackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C, Monte Carlo samples (M)=102026.07 | 81.6 | 7.06 | 0.55 | |
| DALorRABackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C, Monte Carlo samples (M)=102026.07 | 81.6 | 6.23 | 0.54 | |
| MLEBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C2026.07 | 81.48 | 14.45 | 1.07 | |
| MCDBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C, Monte Carlo samples (M)=102026.07 | 81.42 | 13.63 | 1.03 | |
| TFBBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C, Monte Carlo samples (M)=102026.07 | 81.12 | 7.12 | 0.6 | |
| PoLAR-VBLL (w/o LA)Backbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 81.11 | 7.65 | 0.56 | |
| PoLAR-VBLLBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 81.11 | 4.55 | 0.53 | |
| PoLAR-VBLLBackbone=Llama-3.1-8B, Fine-tuning Steps=5000, Training Dataset=OBQA2026.04 | 81.11 | 4.55 | 0.53 | |
| LABackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C2026.07 | 81.08 | 7.83 | 1.17 | |
| PoLAR-MLEBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 80.97 | 10.84 | 0.68 | |
| PoLAR-LABackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 80.97 | 7.09 | 0.56 | |
| PoLAR-LA-LLBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 80.97 | 9.45 | 0.57 | |
| TFB-LLBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 80.97 | 7.86 | 0.58 | |
| TFBBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 80.9 | 5.1 | 0.54 | |
| PoLAR-BLoBBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 80.34 | 6.52 | 0.55 | |
| C-LoRABackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 80.07 | 13.94 | 0.83 | |
| MAPBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C2026.07 | 79.98 | 16.46 | 1.19 | |
| ENSBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C2026.07 | 79.62 | 11.37 | 0.72 | |
| MLEBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 79.33 | 13.89 | 0.9 | |
| LABackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 79.33 | 5.85 | 0.54 | |
| MLEBackbone=Llama-3.1-8B, Fine-tuning Steps=5000, Training Dataset=OBQA2026.04 | 79.33 | 13.89 | 0.9 | |
| BLoBBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 79.1 | 4.87 | 0.55 | |
| BLoBBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->ARC-C, Monte Carlo samples (M)=102026.07 | 78.68 | 7.02 | 0.61 |