Compositional Generalization on SCAN
23.44LengthGRPO-Composite
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GRPO-CompositeBase Model=Llama-3.1-8B-Instruct2026.05 | 23.44 | 70.74 | |
| GRPO-BinaryBase Model=Qwen-2.5-7B-Instruct2026.05 | 22.68 | 79.13 | |
| GRPO-CompositeBase Model=Qwen-2.5-7B-Instruct2026.05 | 21.81 | 78.76 | |
| GRPO-BinaryBase Model=Llama-3.1-8B-Instruct2026.05 | 18.49 | 71.52 | |
| SFTBase Model=Qwen-2.5-7B-Instruct2026.05 | 18.41 | 78.06 | |
| SFTBase Model=Llama-3.1-8B-Instruct2026.05 | 11.88 | 69.03 |