Reward Model Transfer on UltraFeedback (UF)
7.93AOGANCHOR
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ANCHORTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=H3, Evaluation Setting=Zero-shot transfer2026.05 | 7.93 | — | — | 7.93 | |
| Conf.-basedTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=H3, Evaluation Setting=Zero-shot transfer2026.05 | 7.64 | — | — | 5.52 | |
| SEAMTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=H3, Evaluation Setting=Zero-shot transfer2026.05 | 6.25 | — | — | -9.42 | |
| SEAMTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=H3, Evaluation Setting=Zero-shot transfer2026.05 | 5.79 | — | — | -5.8 | |
| ANCHORTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=AH, Evaluation Setting=Zero-shot transfer2026.05 | 5.73 | — | — | 5.73 | |
| ANCHORTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=AH, Evaluation Setting=Zero-shot transfer2026.05 | 5.61 | — | — | 5.61 | |
| SEAMTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=AH, Evaluation Setting=Zero-shot transfer2026.05 | 5.49 | — | — | 2.07 | |
| NaiveTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=AH, Evaluation Setting=Zero-shot transfer2026.05 | 5.15 | — | — | 5.15 | |
| NaiveTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=AH, Evaluation Setting=Zero-shot transfer2026.05 | 4.98 | — | — | 4.98 | |
| Conf.-basedTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=AH, Evaluation Setting=Zero-shot transfer2026.05 | 4.92 | — | — | 4.92 | |
| NaiveTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=H3, Evaluation Setting=Zero-shot transfer2026.05 | 4.8 | — | — | 4.8 | |
| Conf.-basedTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=AH, Evaluation Setting=Zero-shot transfer2026.05 | 4.74 | — | — | 4.74 | |
| ANCHORTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=H3, Evaluation Setting=Zero-shot transfer2026.05 | 4.69 | — | — | 4.69 | |
| Conf.-basedTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=H3, Evaluation Setting=Zero-shot transfer2026.05 | 4.4 | — | — | 2.1 | |
| NaiveTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=H3, Evaluation Setting=Zero-shot transfer2026.05 | 0.17 | — | — | -2.89 | |
| SEAMTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=AH, Evaluation Setting=Zero-shot transfer2026.05 | -0.46 | — | — | -13.5 | |
| ANCHORTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=UF, Evaluation Setting=In-distribution2026.05 | — | 97.9 | 2.32 | — | |
| ANCHORTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=UF, Evaluation Setting=In-distribution2026.05 | — | 189.1 | 0.87 | — | |
| Conf.-basedTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=UF, Evaluation Setting=In-distribution2026.05 | — | 41.8 | 0.99 | — | |
| Conf.-basedTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=UF, Evaluation Setting=In-distribution2026.05 | — | -39.1 | -0.18 | — | |
| NaiveTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=UF, Evaluation Setting=In-distribution2026.05 | — | 73.4 | 1.74 | — | |
| NaiveTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=UF, Evaluation Setting=In-distribution2026.05 | — | -89.1 | -0.41 | — | |
| SEAMTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=UF, Evaluation Setting=In-distribution2026.05 | — | -280.6 | -6.65 | — | |
| SEAMTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=UF, Evaluation Setting=In-distribution2026.05 | — | -517.4 | -2.38 | — |