Response Diversity on Anthropic HH-RLHF
82.5Preference CoverageEvoPref
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| EvoPreftype=Multi-Objective EC, population size=32, generations=50, LoRA rank=16, LoRA alpha=322026.05 | 82.5 | 0.297 | 11 | 0.84 | |
| SMS-EMOAtype=Multi-Objective EC, mu=32, LoRA rank=16, LoRA alpha=322026.05 | 78.2 | 0.327 | 14.1 | 0.82 | |
| MOEA/Dtype=Multi-Objective EC, weight vectors=32, aggregation=Tchebycheff, LoRA rank=16, LoRA alpha=322026.05 | 76.9 | 0.342 | 15.5 | 0.8 | |
| CMA-EStype=Single-Objective EC, mu=32, LoRA rank=16, LoRA alpha=322026.05 | 72.5 | 0.361 | 18.1 | 0.71 | |
| ORPOoptimizer=AdamW, learning rate=5 × 10−5, LoRA rank=16, LoRA alpha=322026.05 | 70 | 0.389 | 20.6 | — | |
| DPObeta=0.1, optimizer=AdamW, learning rate=5 × 10−5, LoRA rank=16, LoRA alpha=322026.05 | 67.1 | 0.414 | 23.3 | — |