Compositional Generalization on COGS
83.9Exact Match AccuracyGRPO-Binary
Evaluation results
| Method | Links | |
|---|---|---|
| GRPO-BinaryBase Model=Llama-3.1-8B-Instruct2026.05 | 83.9 | |
| GRPO-BinaryBase Model=Qwen-2.5-7B-Instruct2026.05 | 83.23 | |
| GRPO-CompositeBase Model=Qwen-2.5-7B-Instruct2026.05 | 83.15 | |
| GRPO-CompositeBase Model=Llama-3.1-8B-Instruct2026.05 | 82.88 | |
| SFTBase Model=Qwen-2.5-7B-Instruct2026.05 | 82.09 | |
| SFTBase Model=Llama-3.1-8B-Instruct2026.05 | 81.12 |