Pairwise LLM-judge evaluation on Arena-Hard and WildBench (combined)
57.7Win RateComb-AT (High)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Comb-AT (High)Compared against=Baseline (no RLHF), KL penalty coefficient (beta_KL)=0.012026.01 | 57.7 | 5 | 37.3 | 0.34 | |
| Comb-AT (Med)Compared against=Baseline (no RLHF), KL penalty coefficient (beta_KL)=0.012026.01 | 57.5 | 4.1 | 38.4 | 0.34 | |
| Base RMCompared against=Baseline (no RLHF), KL penalty coefficient (beta_KL)=0.012026.01 | 55.5 | 5.3 | 39.2 | 0.29 | |
| Comb-AT (Med)Compared against=Baseline (no RLHF), KL penalty coefficient (beta_KL)=0.022026.01 | 55.4 | 7.2 | 37.4 | 0.32 | |
| Comb-AT (High)Compared against=Baseline (no RLHF), KL penalty coefficient (beta_KL)=0.022026.01 | 55.1 | 7.8 | 37.1 | 0.32 | |
| Base RMCompared against=Baseline (no RLHF), KL penalty coefficient (beta_KL)=0.022026.01 | 52.5 | 7.3 | 40.3 | 0.21 | |
| Comb-AT (High)Compared against=Base RM, KL penalty coefficient (beta_KL)=0.012026.01 | 46.5 | 11 | 42.5 | 0.07 | |
| Comb-AT (Med)Compared against=Base RM, KL penalty coefficient (beta_KL)=0.012026.01 | 46.3 | 11.1 | 42.6 | 0.07 | |
| Comb-AT (Med)Compared against=Base RM, KL penalty coefficient (beta_KL)=0.022026.01 | 46.1 | 14 | 39.9 | 0.1 | |
| Comb-AT (High)Compared against=Base RM, KL penalty coefficient (beta_KL)=0.022026.01 | 45.8 | 15.3 | 38.9 | 0.1 | |
| Comb-AT (Med)Compared against=Comb-AT (High), KL penalty coefficient (beta_KL)=0.012026.01 | 45.6 | 11 | 43.4 | 0.04 | |
| Comb-AT (High)Compared against=Comb-AT (Med), KL penalty coefficient (beta_KL)=0.022026.01 | 43.2 | 14.7 | 42.1 | 0.02 |