List your company
ResearchFormerly WizWand

Question Answering on RESTOR

64.8AccuracyIdeal
19.539231.289643.0454.7904Jun 26, 2025
Updated 6mo ago

Evaluation results

MethodLinks
IdealBackbone=Llama-3.1-8B-Instruct2025.0664.8
MSAinstructBackbone=Llama-3.1-8B-Instruct2025.0663.95
MSAbaseBackbone=Llama-3.1-8B-Instruct2025.0659.4
IdealBackbone=OLMo-2-7B2025.0649.76
SatImpBackbone=Llama-3.1-8B-Instruct2025.0649.19
NPOBackbone=Llama-3.1-8B-Instruct2025.0648.45
MSAlastBackbone=OLMo-2-7B, Checkpoint=Last2025.0647.8
MSA3859BBackbone=OLMo-2-7B, Checkpoint=3859B tokens2025.0647.64
MSA3691BBackbone=OLMo-2-7B, Checkpoint=3691B tokens2025.0647.27
MSA2207BBackbone=OLMo-2-7B, Checkpoint=2207B tokens2025.0646.21
MSA500BBackbone=OLMo-2-7B, Checkpoint=500B tokens2025.0645.67
Task VectorBackbone=Llama-3.1-8B-Instruct2025.0644.5
TargetBackbone=Llama-3.1-8B-Instruct2025.0644.31
RMUBackbone=Llama-3.1-8B-Instruct2025.0641.47
SatImpBackbone=OLMo-2-7B2025.0640.25
Task VectorBackbone=OLMo-2-7B2025.0638.47
TargetBackbone=OLMo-2-7B2025.0637.6
RMUBackbone=OLMo-2-7B2025.0636
NPOBackbone=OLMo-2-7B2025.0634.73
GradDiffBackbone=Llama-3.1-8B-Instruct2025.0626.08
GradDiffBackbone=OLMo-2-7B2025.0621.28