Semantic Parsing on CFQ (MCD metrics)
92.1MCD1 ScoreGRPO-Composite
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GRPO-CompositeBase Model=Llama-3.1-8B-Instruct2026.05 | 92.1 | 76.16 | 83.44 | 83.9 | |
| GRPO-BinaryBase Model=Llama-3.1-8B-Instruct2026.05 | 91.35 | 76.44 | 81.46 | 83.08 | |
| SFTBase Model=Llama-3.1-8B-Instruct2026.05 | 91.04 | 74.85 | 77.42 | 81.1 | |
| GRPO-BinaryBase Model=Qwen-2.5-7B-Instruct2026.05 | 89.23 | 75.73 | 76.54 | 80.5 | |
| GRPO-CompositeBase Model=Qwen-2.5-7B-Instruct2026.05 | 89.16 | 77.33 | 75.86 | 80.78 | |
| SFTBase Model=Qwen-2.5-7B-Instruct2026.05 | 88.52 | 73.98 | 73.67 | 78.72 |