Question Answering on RESTOR
64.8AccuracyIdeal
Evaluation Results
| Method | Links | |
|---|---|---|
| IdealBackbone=Llama-3.1-8B-Instruct2025.06 | 64.8 | |
| MSAinstructBackbone=Llama-3.1-8B-Instruct2025.06 | 63.95 | |
| MSAbaseBackbone=Llama-3.1-8B-Instruct2025.06 | 59.4 | |
| IdealBackbone=OLMo-2-7B2025.06 | 49.76 | |
| SatImpBackbone=Llama-3.1-8B-Instruct2025.06 | 49.19 | |
| NPOBackbone=Llama-3.1-8B-Instruct2025.06 | 48.45 | |
| MSAlastBackbone=OLMo-2-7B, Checkpoint=Last2025.06 | 47.8 | |
| MSA3859BBackbone=OLMo-2-7B, Checkpoint=3859B tokens2025.06 | 47.64 | |
| MSA3691BBackbone=OLMo-2-7B, Checkpoint=3691B tokens2025.06 | 47.27 | |
| MSA2207BBackbone=OLMo-2-7B, Checkpoint=2207B tokens2025.06 | 46.21 | |
| MSA500BBackbone=OLMo-2-7B, Checkpoint=500B tokens2025.06 | 45.67 | |
| Task VectorBackbone=Llama-3.1-8B-Instruct2025.06 | 44.5 | |
| TargetBackbone=Llama-3.1-8B-Instruct2025.06 | 44.31 | |
| RMUBackbone=Llama-3.1-8B-Instruct2025.06 | 41.47 | |
| SatImpBackbone=OLMo-2-7B2025.06 | 40.25 | |
| Task VectorBackbone=OLMo-2-7B2025.06 | 38.47 | |
| TargetBackbone=OLMo-2-7B2025.06 | 37.6 | |
| RMUBackbone=OLMo-2-7B2025.06 | 36 | |
| NPOBackbone=OLMo-2-7B2025.06 | 34.73 | |
| GradDiffBackbone=Llama-3.1-8B-Instruct2025.06 | 26.08 | |
| GradDiffBackbone=OLMo-2-7B2025.06 | 21.28 |