Medical Question Answering on Out-of-domain medical QA History, Engineering, Law (test)
50.4History AccuracyKaFT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| KaFTBackbone=Mistral-7B2025.05 | 50.4 | 60 | 51.6 | 54 | 6.93 | |
| Vanilla SFTBackbone=Mistral-7B2025.05 | 46 | 59.2 | 50 | 51.73 | 4.66 | |
| No-conflictBackbone=Mistral-7B2025.05 | 45.2 | 61.2 | 49.6 | 52 | 4.93 | |
| Self-aligningBackbone=Mistral-7B2025.05 | 44.4 | 56.8 | 49.2 | 50.13 | 3.06 | |
| BaseBackbone=Mistral-7B2025.05 | 41.2 | 53.2 | 46.8 | 47.07 | — | |
| No-conflictBackbone=LLAMA3-3B2025.05 | 40.8 | 58 | 48 | 48.93 | 6.66 | |
| KaFTBackbone=LLAMA3-3B2025.05 | 40.4 | 58.4 | 49.6 | 49.47 | 7.2 | |
| Vanilla SFTBackbone=LLAMA3-3B2025.05 | 40 | 56.4 | 46.4 | 47.6 | 5.33 | |
| Self-aligningBackbone=LLAMA3-3B2025.05 | 39.6 | 56.4 | 48 | 48 | 5.73 | |
| BaseBackbone=LLAMA3-3B2025.05 | 33.6 | 54.8 | 38.4 | 42.27 | — |