LLM Alignment on HH-RLHF (test)
0.87DiversityRE-Control
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| RE-ControlModel=Hermes3-8B2026.04 | 0.87 | 0.55 | -4.321 | — | 85.5 | 84 | 73.1 | |
| Pref-CTRL (Margin + Regularizer)Model=Hermes3-8B2026.04 | 0.87 | 0.56 | -4.268 | — | 85.7 | 84.3 | 73.6 | |
| Pref-CTRL (Margin)Model=Hermes3-8B2026.04 | 0.86 | 0.55 | -4.301 | — | 86.7 | 83.7 | 73.2 | |
| DPOModel=Hermes3-8B2026.04 | 0.85 | 0.55 | -4.297 | — | 86.8 | 78.7 | 75.8 | |
| RE-CONTROL + PromptingBackbone=Vicuna-7B, Inference time (hour)=0.932024.06 | 0.83 | 0.577 | 6.267 | 80.3 | — | — | — | |
| RE-CONTROLBackbone=Vicuna-7B, Inference time (hour)=0.852024.06 | 0.824 | 0.579 | 6.214 | 75.6 | — | — | — | |
| Static REBackbone=Vicuna-7B, Inference time (hour)=0.652024.06 | 0.818 | 0.568 | 5.907 | 64.3 | — | — | — | |
| PromptingBackbone=Vicuna-7B, Inference time (hour)=0.692024.06 | 0.817 | 0.57 | 5.913 | 66 | — | — | — | |
| BaseBackbone=Vicuna-7B, Inference time (hour)=0.602024.06 | 0.816 | 0.568 | 5.894 | 57.6 | — | — | — | |
| CD prefix + PromptingBackbone=Vicuna-7B, Inference time (hour)=47.162024.06 | 0.812 | 0.593 | 6.12 | 74.3 | — | — | — | |
| CDBackbone=Vicuna-7B, Inference time (hour)=47.432024.06 | 0.806 | 0.608 | 5.458 | 72.3 | — | — | — | |
| CD prefixBackbone=Vicuna-7B, Inference time (hour)=32.132024.06 | 0.805 | 0.576 | 6.105 | 74.6 | — | — | — | |
| Pref-CTRL (Margin)Model=Vicuna-7B2026.04 | 0.75 | 0.53 | -5.358 | — | 80.7 | 82.5 | 72.4 | |
| PromptingBackbone=Falcon-7B, Inference time (hour)=0.592024.06 | 0.746 | 0.62 | 4.01 | 52.3 | — | — | — | |
| RE-CONTROL + PromptingBackbone=Falcon-7B, Inference time (hour)=2.002024.06 | 0.741 | 0.619 | 4.083 | 62.6 | — | — | — | |
| RE-ControlModel=Vicuna-7B2026.04 | 0.72 | 0.54 | -5.408 | — | 81.9 | 85.4 | 73.3 | |
| DPOModel=Vicuna-7B2026.04 | 0.71 | 0.55 | -5.591 | — | 85.1 | 89 | 81.2 | |
| BaseBackbone=Falcon-7B, Inference time (hour)=0.672024.06 | 0.705 | 0.613 | 3.439 | 42.3 | — | — | — | |
| Pref-CTRL (Margin + Regularizer)Model=Vicuna-7B2026.04 | 0.7 | 0.55 | -5.288 | — | 82.9 | 85.6 | 74.6 | |
| RE-CONTROLBackbone=Falcon-7B, Inference time (hour)=1.932024.06 | 0.699 | 0.615 | 3.512 | 58 | — | — | — | |
| Static REBackbone=Falcon-7B, Inference time (hour)=0.562024.06 | 0.698 | 0.61 | 3.449 | 52.6 | — | — | — | |
| CD prefixBackbone=Falcon-7B, Inference time (hour)=48.132024.06 | 0.648 | 0.575 | 4.397 | 49.6 | — | — | — | |
| CD prefix + PromptingBackbone=Falcon-7B, Inference time (hour)=47.872024.06 | 0.571 | 0.638 | 3.619 | 51.6 | — | — | — | |
| MAVISBase Model=Llama-7B, Baseline Compared Against=MOD2025.08 | — | — | — | 56.5 | — | — | — | |
| MAVISBase Model=Llama-7B, Baseline Compared Against=RSoup2025.08 | — | — | — | 67.5 | — | — | — | |
| MAVISBase Model=Llama-7B, Baseline Compared Against=MOODPO2025.08 | — | — | — | 35.5 | — | — | — | |
| MAVISBase Model=Llama-13B, Baseline Compared Against=MOD2025.08 | — | — | — | 59.5 | — | — | — | |
| MAVISBase Model=Llama-13B, Baseline Compared Against=RSoup2025.08 | — | — | — | 62 | — | — | — | |
| MAVISBase Model=Llama-13B, Baseline Compared Against=MOODPO2025.08 | — | — | — | 62 | — | — | — |