Reward Model Transfer on RAIL
4.62AOGANCHOR
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| ANCHORTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=PKU, Evaluation Setting=Zero-shot transfer2026.05 | 4.62 | — | — | 4.62 | |
| NaiveTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=PKU, Evaluation Setting=Zero-shot transfer2026.05 | 2.26 | — | — | 2.26 | |
| NaiveTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=PKU, Evaluation Setting=Zero-shot transfer2026.05 | 1.85 | — | — | 1.85 | |
| ANCHORTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=PKU, Evaluation Setting=Zero-shot transfer2026.05 | 1.85 | — | — | 1.85 | |
| ANCHORTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=AHar, Evaluation Setting=Zero-shot transfer2026.05 | 1.65 | — | — | 1.65 | |
| NaiveTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=AHar, Evaluation Setting=Zero-shot transfer2026.05 | 0.52 | — | — | 0.52 | |
| ANCHORTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=AHar, Evaluation Setting=Zero-shot transfer2026.05 | 0.31 | — | — | 0.31 | |
| NaiveTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=AHar, Evaluation Setting=Zero-shot transfer2026.05 | 0 | — | — | 0 | |
| ANCHORTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=RAIL, Evaluation Setting=In-distribution2026.05 | — | 77.8 | 1.44 | — | |
| ANCHORTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=RAIL, Evaluation Setting=In-distribution2026.05 | — | 0 | 0 | — | |
| NaiveTransfer Pair=Llama-3.2-1B-Instruct -> Llama-3.1-8B-Instruct, Source Domain=RAIL, Evaluation Setting=In-distribution2026.05 | — | 55.7 | 1.03 | — | |
| NaiveTransfer Pair=Qwen3-1.7B -> Qwen3-8B, Source Domain=RAIL, Evaluation Setting=In-distribution2026.05 | — | -19.1 | -0.51 | — |