Reward Model Transfer on PKU-SafeRLHF
2.46AOGANCHOR
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ANCHORTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=RAIL, Evaluation Setting=Zero-shot transfer2026.05 | 2.46 | — | — | 2.46 | |
| ANCHORTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=AHar, Evaluation Setting=Zero-shot transfer2026.05 | 2.38 | — | — | 2.38 | |
| ANCHORTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=RAIL, Evaluation Setting=Zero-shot transfer2026.05 | 2.37 | — | — | 2.37 | |
| NaiveTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=AHar, Evaluation Setting=Zero-shot transfer2026.05 | 2.05 | — | — | 2.05 | |
| NaiveTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=RAIL, Evaluation Setting=Zero-shot transfer2026.05 | 2.04 | — | — | 2.04 | |
| NaiveTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=AHar, Evaluation Setting=Zero-shot transfer2026.05 | 1.97 | — | — | 1.97 | |
| NaiveTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=RAIL, Evaluation Setting=Zero-shot transfer2026.05 | 1.72 | — | — | 1.21 | |
| ANCHORTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=AHar, Evaluation Setting=Zero-shot transfer2026.05 | 0.66 | — | — | 0.66 | |
| ANCHORTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=PKU, Evaluation Setting=In-distribution2026.05 | — | 163.4 | 5.49 | — | |
| ANCHORTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=PKU, Evaluation Setting=In-distribution2026.05 | — | 2.7 | 2.21 | — | |
| NaiveTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=PKU, Evaluation Setting=In-distribution2026.05 | — | 146.1 | 4.91 | — | |
| NaiveTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=PKU, Evaluation Setting=In-distribution2026.05 | — | 1 | 0.82 | — |