List your company
ResearchFormerly WizWand

Common-sense reasoning on Chem OOD Large Shift

50AccuracyDALorRA
42.35644.340546.32548.3095Apr 3, 2026Apr 18, 2026May 3, 2026May 18, 2026Jun 2, 2026Jun 17, 2026Jul 2, 2026
Updated 4mo ago

Evaluation results

MethodLinks
DALorRABackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem, Monte Carlo samples (M)=102026.075014.581.25
ENSBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem2026.0749.6518.921.4
PoLAR-VBLL (w/o LA)Backbone=Llama-3.1-8B, Training dataset=OBQA2026.0449.315.051.21
PoLAR-VBLLBackbone=Llama-3.1-8B, Training dataset=OBQA2026.0449.310.31.16
PoLAR-VBLLBackbone=Llama-3.1-8B, Fine-tuning Steps=5000, Training Dataset=OBQA2026.0449.310.31.16
PoLAR-MLEBackbone=Llama-3.1-8B, Training dataset=OBQA2026.0448.3326.331.56
PoLAR-LABackbone=Llama-3.1-8B, Training dataset=OBQA2026.0448.3311.481.18
PoLAR-LA-LLBackbone=Llama-3.1-8B, Training dataset=OBQA2026.0448.3314.891.42
LABackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem2026.0748.2614.491.27
MLEBackbone=Llama-3.1-8B, Training dataset=OBQA2026.044828.731.75
LABackbone=Llama-3.1-8B, Training dataset=OBQA2026.044811.691.19
MLEBackbone=Llama-3.1-8B, Fine-tuning Steps=5000, Training Dataset=OBQA2026.044828.731.75
MCDBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem, Monte Carlo samples (M)=102026.0747.9230.911.91
C-LoRABackbone=Llama-3.1-8B, Training dataset=OBQA2026.0447.6727.991.69
TFB-LLBackbone=Llama-3.1-8B, Training dataset=OBQA2026.0447.3318.751.27
TFBBackbone=Llama-3.1-8B, Training dataset=OBQA2026.0446.8717.831.23
PoLAR-BLoBBackbone=Llama-3.1-8B, Training dataset=OBQA2026.0446.1820.291.26
MLEBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem2026.0745.8332.461.91
BLoBBackbone=Llama-3.1-8B, Training dataset=OBQA2026.0445.6713.231.18
C-LoRABackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem, Monte Carlo samples (M)=102026.0745.6426.481.66
BLoBBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem, Monte Carlo samples (M)=102026.0743.7514.791.23
MAPBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem2026.0743.434.791.97
TFBBackbone=Llama3.1-8B, Fine-tuning method=LoRA, Evaluation protocol=OBQA->Chem, Monte Carlo samples (M)=102026.0742.6515.321.34