RLHF Alignment on UltraFeedback In-domain v1 (test)
81Win RateGEB-π
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GEB-πBackbone=Mistral-Instruct-v0.3, Divergence=KL, Alpha=12025.09 | 81 | 0.7542 | |
| GEB-1/πBackbone=Mistral-Instruct-v0.3, Divergence=KL, Alpha=12025.09 | 80 | 0.7554 | |
| XPOBackbone=Mistral-Instruct-v0.3, Divergence=KL, Alpha=12025.09 | 79.71 | 0.7492 | |
| GEB-arctanh(π − 1)Backbone=Mistral-Instruct-v0.3, Divergence=KL, Alpha=12025.09 | 79.71 | 0.7559 | |
| VPOBackbone=Mistral-Instruct-v0.3, Divergence=KL, Alpha=12025.09 | 78.57 | 0.7426 | |
| f-DPOBackbone=Mistral-Instruct-v0.3, Divergence=KL, Alpha=12025.09 | 78.42 | 0.748 | |
| FEBBackbone=Mistral-Instruct-v0.3, Divergence=KL, Alpha=12025.09 | 78.42 | 0.748 | |
| SELMBackbone=Mistral-Instruct-v0.3, Divergence=KL, Alpha=12025.09 | 77.56 | 0.753 | |
| GEB-arctanh(π − 1)Backbone=Mistral-Instruct-v0.3, Divergence=Hellinger, Alpha=0.52025.09 | 75.69 | 0.6614 | |
| GEB-πBackbone=Mistral-Instruct-v0.3, Divergence=Hellinger, Alpha=0.52025.09 | 75.48 | 0.6641 | |
| GEB-arctanh(π − 1)Backbone=LLaMA-3-8B-SFT, Divergence=KL, Alpha=12025.09 | 74.98 | 0.808 | |
| GEB-1/πBackbone=LLaMA-3-8B-SFT, Divergence=KL, Alpha=12025.09 | 74.76 | 0.8102 | |
| GEB-πBackbone=LLaMA-3-8B-SFT, Divergence=KL, Alpha=12025.09 | 74.34 | 0.8156 | |
| SELMBackbone=LLaMA-3-8B-SFT, Divergence=KL, Alpha=12025.09 | 74.19 | 0.8126 | |
| GEB-1/πBackbone=Mistral-Instruct-v0.3, Divergence=Hellinger, Alpha=0.52025.09 | 73.97 | 0.6541 | |
| GEB-arctanh(π − 1)Backbone=LLaMA-3-8B-SFT, Divergence=Hellinger, Alpha=0.52025.09 | 73.26 | 0.7877 | |
| f-DPOBackbone=LLaMA-3-8B-SFT, Divergence=KL, Alpha=12025.09 | 73.11 | 0.805 | |
| FEBBackbone=LLaMA-3-8B-SFT, Divergence=KL, Alpha=12025.09 | 73.11 | 0.805 | |
| f-DPOBackbone=Mistral-Instruct-v0.3, Divergence=Hellinger, Alpha=0.52025.09 | 72.69 | 0.6536 | |
| XPOBackbone=LLaMA-3-8B-SFT, Divergence=KL, Alpha=12025.09 | 72.4 | 0.8119 | |
| GEB-arctanh(π − 1)Backbone=LLaMA-3-8B-SFT, Divergence=Average2025.09 | 72.38 | 0.7842 | |
| GEB-1/πBackbone=LLaMA-3-8B-SFT, Divergence=Hellinger, Alpha=0.52025.09 | 72.25 | 0.7859 | |
| GEB-1/πBackbone=LLaMA-3-8B-SFT, Divergence=Average2025.09 | 71.73 | 0.7851 | |
| GEB-πBackbone=LLaMA-3-8B-SFT, Divergence=Hellinger, Alpha=0.52025.09 | 71.68 | 0.784 | |
| VPOBackbone=LLaMA-3-8B-SFT, Divergence=KL, Alpha=12025.09 | 71.61 | 0.7971 | |
| FEBBackbone=Mistral-Instruct-v0.3, Divergence=Hellinger, Alpha=0.52025.09 | 71.54 | 0.6525 | |
| GEB-πBackbone=LLaMA-3-8B-SFT, Divergence=Average2025.09 | 71.23 | 0.7892 | |
| f-DPOBackbone=LLaMA-3-8B-SFT, Divergence=Hellinger, Alpha=0.52025.09 | 71.11 | 0.7859 | |
| f-DPOBackbone=LLaMA-3-8B-SFT, Divergence=Average2025.09 | 70.53 | 0.7829 | |
| FEBBackbone=LLaMA-3-8B-SFT, Divergence=Average2025.09 | 69.74 | 0.7751 | |
| GEB-πBackbone=Mistral-Instruct-v0.3, Divergence=Average2025.09 | 69.39 | 0.672 | |
| GEB-arctanh(π − 1)Backbone=Mistral-Instruct-v0.3, Divergence=Average2025.09 | 69.39 | 0.6721 | |
| GEB-arctanh(π − 1)Backbone=LLaMA-3-8B-SFT, Divergence=f-KL, Alpha=02025.09 | 68.89 | 0.7569 | |
| GEB-1/πBackbone=Mistral-Instruct-v0.3, Divergence=Average2025.09 | 68.74 | 0.6715 | |
| FEBBackbone=LLaMA-3-8B-SFT, Divergence=Hellinger, Alpha=0.52025.09 | 68.17 | 0.7591 | |
| GEB-1/πBackbone=LLaMA-3-8B-SFT, Divergence=f-KL, Alpha=02025.09 | 68.17 | 0.7591 | |
| FEBBackbone=LLaMA-3-8B-SFT, Divergence=f-KL, Alpha=02025.09 | 67.95 | 0.7611 | |
| GEB-πBackbone=LLaMA-3-8B-SFT, Divergence=f-KL, Alpha=02025.09 | 67.67 | 0.7681 | |
| f-DPOBackbone=Mistral-Instruct-v0.3, Divergence=Average2025.09 | 67.4 | 0.6645 | |
| f-DPOBackbone=LLaMA-3-8B-SFT, Divergence=f-KL, Alpha=02025.09 | 67.38 | 0.7579 | |
| FEBBackbone=Mistral-Instruct-v0.3, Divergence=Average2025.09 | 65.83 | 0.6644 | |
| GEB-arctanh(π − 1)Backbone=Mistral-Instruct-v0.3, Divergence=f-KL, Alpha=02025.09 | 52.76 | 0.5989 | |
| GEB-1/πBackbone=Mistral-Instruct-v0.3, Divergence=f-KL, Alpha=02025.09 | 52.26 | 0.6051 | |
| GEB-πBackbone=Mistral-Instruct-v0.3, Divergence=f-KL, Alpha=02025.09 | 51.68 | 0.5976 | |
| f-DPOBackbone=Mistral-Instruct-v0.3, Divergence=f-KL, Alpha=02025.09 | 51.11 | 0.5918 | |
| FEBBackbone=Mistral-Instruct-v0.3, Divergence=f-KL, Alpha=02025.09 | 47.53 | 0.5928 |