ResearchBenchmarksConversational Assistant on HH-RLHFConversational Assistant on HH-RLHFFollow0.5RewardRLOO-0.436-0.1930.050.293Mar 5, 2025Evaluation resultsMethodMethodLinksRewardExecution Time (s)Reward RatePeak Memory (GB)RLOOModel=Llama 3.2 1B, Ha...Model=Llama 3.2 1B, Hardware=4 x A100 (40GB VRAM)2025.030.550.0017—ZOPrOModel=Llama 3.2 1B, Ha...Model=Llama 3.2 1B, Hardware=4 x A100 (40GB VRAM)2025.030.251200.0031—PPOModel=Llama 3.2 1B, Ha...Model=Llama 3.2 1B, Hardware=4 x A100 (40GB VRAM)2025.03-0.4530-0.0012—