ResearchBenchmarksInstruction Following on UltraChatInstruction Following on UltraChatFollow67.8RM ScoreSFT + TTL64.05665.0286666.972May 8, 2026Evaluation resultsMethodMethodLinksRM ScoreWin RateToxicitySFT + TTLBase LLM=Qwen2.5-7B-In...Base LLM=Qwen2.5-7B-Instruct, Training Configuration=LoRA (r=16)2026.0567.858.438Base SFTBase LLM=Qwen2.5-7B-In...Base LLM=Qwen2.5-7B-Instruct, Training Configuration=LoRA (r=16)2026.0564.2—45