ResearchBenchmarksSummarization on TL;DR (Winrate, RM, KL)Follow50.5WinrateHyPO41.836844.085946.33548.5841Jun 3, 2024Evaluation ResultsMethodMethodLinksWinrateRM ScoreKL Divergence (π vs πref)HyPOModel size=2.8BModel size=2.8B2024.0650.52.5148.98HyPOModel size=1.4BModel size=1.4B2024.0646.440.3727.07DPOModel size=2.8BModel size=2.8B2024.0644.392.4368.95DPOModel size=1.4BModel size=1.4B2024.0642.170.1644.9