Common-sense reasoning on Chem OOD Large Shift
50AccuracyDALorRA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| DALorRABackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem, Monte Carlo samples (M)=102026.07 | 50 | 14.58 | 1.25 | |
| ENSBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem2026.07 | 49.65 | 18.92 | 1.4 | |
| PoLAR-VBLL (w/o LA)Backbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 49.3 | 15.05 | 1.21 | |
| PoLAR-VBLLBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 49.3 | 10.3 | 1.16 | |
| PoLAR-VBLLBackbone=Llama-3.1-8B, Fine-tuning Steps=5000, Training Dataset=OBQA2026.04 | 49.3 | 10.3 | 1.16 | |
| PoLAR-MLEBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 48.33 | 26.33 | 1.56 | |
| PoLAR-LABackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 48.33 | 11.48 | 1.18 | |
| PoLAR-LA-LLBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 48.33 | 14.89 | 1.42 | |
| LABackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem2026.07 | 48.26 | 14.49 | 1.27 | |
| MLEBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 48 | 28.73 | 1.75 | |
| LABackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 48 | 11.69 | 1.19 | |
| MLEBackbone=Llama-3.1-8B, Fine-tuning Steps=5000, Training Dataset=OBQA2026.04 | 48 | 28.73 | 1.75 | |
| MCDBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem, Monte Carlo samples (M)=102026.07 | 47.92 | 30.91 | 1.91 | |
| C-LoRABackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 47.67 | 27.99 | 1.69 | |
| TFB-LLBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 47.33 | 18.75 | 1.27 | |
| TFBBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 46.87 | 17.83 | 1.23 | |
| PoLAR-BLoBBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 46.18 | 20.29 | 1.26 | |
| MLEBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem2026.07 | 45.83 | 32.46 | 1.91 | |
| BLoBBackbone=Llama-3.1-8B, Training dataset=OBQA2026.04 | 45.67 | 13.23 | 1.18 | |
| C-LoRABackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem, Monte Carlo samples (M)=102026.07 | 45.64 | 26.48 | 1.66 | |
| BLoBBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem, Monte Carlo samples (M)=102026.07 | 43.75 | 14.79 | 1.23 | |
| MAPBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem2026.07 | 43.4 | 34.79 | 1.97 | |
| TFBBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem, Monte Carlo samples (M)=102026.07 | 42.65 | 15.32 | 1.34 |