List your company
ResearchFormerly WizWand

Multi-hop Question Answering on MuSiQue (Token-level F1)

38.1Token-level F1PDA-GAM
20.73225.24129.7534.259May 29, 2026
Updated 5mo ago

Evaluation results

MethodLinks
PDA-GAMStudent Model=Qwen3-8B, Number of Preference Deltas (∆)=42026.0538.1
PDA-GAMStudent Model=Qwen3-8B, Number of Preference Deltas (∆)=32026.0537.9
PDA-TIESStudent Model=Qwen3-8B, Number of Preference Deltas (∆)=32026.0537.8
PDA-TIESStudent Model=Qwen3-8B, Number of Preference Deltas (∆)=42026.0537.8
PDA-KnOTSStudent Model=Qwen3-8B, Number of Preference Deltas (∆)=42026.0537
PDA-AveragingStudent Model=Qwen3-8B, Number of Preference Deltas (∆)=42026.0536.9
PDA-GAMStudent Model=Qwen3-8B, Number of Preference Deltas (∆)=22026.0536.1
PDA-TIESStudent Model=Qwen3-8B, Number of Preference Deltas (∆)=22026.0535.6
Search-R1Student Model=Qwen3-8B, Preference Delta Scale=7B over 3B, Number of Preference Deltas (∆)=12026.0534.7
Qwen3Student Model=Qwen3-8B, Preference Delta Scale=4B over 1.7B, Number of Preference Deltas (∆)=12026.0532.8
PDA-GAMStudent Model=Tülu3-8B, Number of Preference Deltas (∆)=42026.0532.8
PDA-GAMStudent Model=Tülu3-8B, Number of Preference Deltas (∆)=32026.0532.4
PDA-TIESStudent Model=Tülu3-8B, Number of Preference Deltas (∆)=42026.0532.1
Joint ORPOStudent Model=Qwen3-8B, Number of Preference Deltas (∆)=42026.0532
PDA-AveragingStudent Model=Tülu3-8B, Number of Preference Deltas (∆)=42026.0532
PDA-TIESStudent Model=Tülu3-8B, Number of Preference Deltas (∆)=32026.0532
PDA-KnOTSStudent Model=Tülu3-8B, Number of Preference Deltas (∆)=42026.0531.9
PDA-TIESStudent Model=Tülu3-8B, Number of Preference Deltas (∆)=22026.0531.6
AceSearcherStudent Model=Qwen3-8B, Preference Delta Scale=7B over 3B, Number of Preference Deltas (∆)=12026.0530.5
PDA-GAMStudent Model=Tülu3-8B, Number of Preference Deltas (∆)=22026.0530.3
Search-R1Student Model=Tülu3-8B, Preference Delta Scale=7B over 3B, Number of Preference Deltas (∆)=12026.0529.5
Llama-3.2Student Model=Qwen3-8B, Preference Delta Scale=3B over 1B, Number of Preference Deltas (∆)=12026.0529.1
Student BaselineStudent Model=Qwen3-8B2026.0528.9
Joint ORPOStudent Model=Tülu3-8B, Number of Preference Deltas (∆)=42026.0528.9
Qwen3Student Model=Tülu3-8B, Preference Delta Scale=4B over 1.7B, Number of Preference Deltas (∆)=12026.0528.5
AceSearcherStudent Model=Tülu3-8B, Preference Delta Scale=7B over 3B, Number of Preference Deltas (∆)=12026.0527.8
Sequential ORPOStudent Model=Qwen3-8B, Number of Preference Deltas (∆)=22026.0527.6
Llama-3.2Student Model=Tülu3-8B, Preference Delta Scale=3B over 1B, Number of Preference Deltas (∆)=12026.0526.2
Student BaselineStudent Model=Tülu3-8B2026.0525.6
Sequential ORPOStudent Model=Tülu3-8B, Number of Preference Deltas (∆)=22026.0521.4