ResearchBenchmarksSummarization on Preference Optimization SummarizationSummarization on Preference Optimization SummarizationFollow0.3RewardZOPrO0.22720.24610.2650.2839Mar 5, 2025Evaluation resultsMethodMethodLinksRewardZOPrOModel=Qwen 2.5, Parame...Model=Qwen 2.5, Parameters=1.5B2025.030.3ZOPrOModel=Gemma 2, Paramet...Model=Gemma 2, Parameters=2.4B2025.030.23