ResearchBenchmarksSummarization on Summarize from FeedbackSummarization on Summarize from FeedbackFollow70RewardPPO12.827.6542.557.35Mar 5, 2025Evaluation resultsMethodMethodLinksRewardTime (m)Reward / TimePeak Memory (GB)PPOModel=Llama 3.2 1B, Ha...Model=Llama 3.2 1B, Hardware=4 x A100 (40GB VRAM)2025.0370490.014—ZOPrOModel=Llama 3.2 1B, Ha...Model=Llama 3.2 1B, Hardware=4 x A100 (40GB VRAM)2025.0317140.012—RLOOModel=Llama 3.2 1B, Ha...Model=Llama 3.2 1B, Hardware=4 x A100 (40GB VRAM)2025.0315420.0035—