Question Answering on PubMedQA Retrieval OS Setting (test)
76.4AccuracyOS-8B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OS-8B2026.04 | 76.4 | — | |
| LoRA FT 2ep 10k contextFine-Tuning=LoRA, Epochs=2, Context Length=10k2026.04 | 58.01 | 55.83 | |
| Full FT 5ep 16k contextFine-Tuning=Full, Epochs=5, Context Length=16k2026.04 | 58.01 | 56.21 | |
| Llama 3.2 3B InstructModel=Llama 3.2, Parameters=3B, Type=Instruct2026.04 | 49.11 | 49.04 | |
| Llama 3.1 8B InstructModel=Llama 3.1, Parameters=8B, Type=Instruct2026.04 | 45.91 | 43.35 |