Common-sense reasoning on Phy OOD Large Shift
48.91AccuracyPoLAR-VBLL (w/o LA)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PoLAR-VBLL (w/o LA)Backbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 48.91 | 15.63 | 1.27 | |
| PoLAR-VBLLBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 48.91 | 11.12 | 1.2 | |
| TFBBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 48.83 | 15.8 | 1.31 | |
| DALorRABackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Phy, Monte Carlo samples (M)=102026.07 | 47.22 | 15.46 | 1.25 | |
| BLoBBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Phy, Monte Carlo samples (M)=102026.07 | 46.96 | 12.34 | 1.28 | |
| PoLAR-BLoBBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 46.88 | 18.43 | 1.21 | |
| TFBBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Phy, Monte Carlo samples (M)=102026.07 | 46.58 | 16.32 | 1.26 | |
| MCDBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Phy, Monte Carlo samples (M)=102026.07 | 46.53 | 33.08 | 2.02 | |
| LABackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Phy2026.07 | 46.18 | 13.17 | 1.28 | |
| TFB-LLBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 45.67 | 20.67 | 1.37 | |
| BLoBBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 45.67 | 11.31 | 1.31 | |
| ENSBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Phy2026.07 | 44.44 | 26.8 | 1.5 | |
| MLEBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 43.33 | 37.02 | 1.94 | |
| LABackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 43.33 | 13.02 | 1.22 | |
| PoLAR-MLEBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 43.33 | 33.22 | 1.61 | |
| PoLAR-LABackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 43.33 | 16.09 | 1.2 | |
| PoLAR-LA-LLBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 43.33 | 13.75 | 1.46 | |
| MLEBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Phy2026.07 | 42.36 | 38.41 | 2.25 | |
| C-LoRABackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 41.33 | 35.25 | 1.84 | |
| C-LoRABackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Phy, Monte Carlo samples (M)=102026.07 | 40.38 | 30.28 | 1.88 | |
| MAPBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Phy2026.07 | 38.54 | 38.5 | 2.32 |