List your company
ResearchFormerly WizWand

Summarization on Summarization (Custom Metrics)

95Win RateCurriculum-RLAIF
15.866436.410756.95577.4993Jun 21, 2024Aug 16, 2024Oct 12, 2024Dec 7, 2024Feb 2, 2025Mar 30, 2025May 26, 2025
Updated 6mo ago

Evaluation results

MethodLinks
Curriculum-RLAIFBase Model=Qwen2.5-32B, Category=Curriculum2025.0595———
External Eval.Base Model=Qwen2.5-32B, Category=Curriculum2025.0593———
Curriculum-RLAIFBase Model=LLaMA-3-8B, Category=Curriculum2025.0592———
Internal Eval.Base Model=Qwen2.5-32B, Category=Curriculum2025.0592———
Implicit Eval. (DPO)Base Model=Qwen2.5-32B, Category=Curriculum2025.0591———
Internal Eval.Base Model=LLaMA-3-8B, Category=Curriculum2025.0590———
Conventional RLAIFBase Model=Qwen2.5-32B, Category=Non-Curriculum2025.0590———
External Eval.Base Model=LLaMA-3-8B, Category=Curriculum2025.0589———
Curriculum-RLAIFBase Model=Gemma-1-2B, Category=Curriculum2025.0588———
Implicit Eval. (DPO)Base Model=LLaMA-3-8B, Category=Curriculum2025.0587———
RLCDBase Model=Qwen2.5-32B, Category=Non-Curriculum2025.0587———
External Eval.Base Model=Gemma-1-2B, Category=Curriculum2025.0586———
CAIBase Model=Qwen2.5-32B, Category=Non-Curriculum2025.0586———
Internal Eval.Base Model=Gemma-1-2B, Category=Curriculum2025.0585———
Conventional RLAIFBase Model=LLaMA-3-8B, Category=Non-Curriculum2025.0585———
Implicit Eval. (DPO)Base Model=Gemma-1-2B, Category=Curriculum2025.0582———
RLCDBase Model=LLaMA-3-8B, Category=Non-Curriculum2025.0581———
Conventional RLAIFBase Model=Gemma-1-2B, Category=Non-Curriculum2025.0580———
CAIBase Model=LLaMA-3-8B, Category=Non-Curriculum2025.0579———
HBAT#Param=13B, PPO=true, DPO=false2024.0678.8126.77-3.516.41
HBAT-Freeze#Param=13B, PPO=true, DPO=false2024.0677.2225.8-3.636.18
RLCDBase Model=Gemma-1-2B, Category=Non-Curriculum2025.0577———
CAIBase Model=Gemma-1-2B, Category=Non-Curriculum2025.0575———
HBAT#Param=13B, PPO=false, DPO=true2024.0674.9224.12-4.055.4
HBAT#Param=7B, PPO=true, DPO=false2024.0672.5226.18-3.825.74
RLHF#Param=13B, PPO=true, DPO=false2024.0671.6724.51-3.965.55
HBAT-Freeze#Param=13B, PPO=false, DPO=true2024.0671.4423.1-5.084.85
HBAT#Param=7B, PPO=false, DPO=true2024.0670.5823.14-4.184.95
DPO#Param=13B, PPO=false, DPO=true2024.0669.423.02-5.394.55
HBAT-Freeze#Param=7B, PPO=true, DPO=false2024.0664.7925.33-4.285.26
RLHF+pt#Param=13B, PPO=true, DPO=false2024.0664.4222.92-5.493.97
HBAT-Freeze#Param=7B, PPO=false, DPO=true2024.0664.1823.01-5.054.45
RLHF#Param=7B, PPO=true, DPO=false2024.0663.825.85-4.274.43
DPO#Param=7B, PPO=false, DPO=true2024.0661.3722.96-5.134.27
SFT#Param=13B, PPO=false, DPO=false2024.0657.9123.27-5.124.01
RLHF+pt#Param=7B, PPO=true, DPO=false2024.0656.2622.25-5.643.74
SFT#Param=7B, PPO=false, DPO=false2024.0653.222.6-5.463.72
SFT+ppo#Param=13B, PPO=true, DPO=false2024.0628.9713.84-5.972.53
SFT+ppo#Param=7B, PPO=true, DPO=false2024.0618.9113.75-5.782.4