Pragmatic Reasoning on ALTPRAG
8.14Reference-Based ScorePRAGREST-GRPO
Evaluation Results
| Method | Links | |
|---|---|---|
| PRAGREST-GRPOBackbone=Qwen3-14B, Decoding=greedy decoding2026.06 | 8.14 | |
| PRAGREST-SFTBackbone=Qwen3-14B, Decoding=greedy decoding2026.06 | 8.06 | |
| Non-counterfactual-GRPOBackbone=Qwen3-14B, Decoding=greedy decoding2026.06 | 7.87 | |
| Non-counterfactual-SFTBackbone=Qwen3-14B, Decoding=greedy decoding2026.06 | 7.8 | |
| InstructBackbone=Qwen3-14B, Decoding=greedy decoding2026.06 | 7.78 | |
| DEEP-LAYER-DPOBackbone=Qwen3-14B, Decoding=greedy decoding2026.06 | 7.71 | |
| PRAGREST-GRPOBackbone=Qwen3-8B, Decoding=greedy decoding2026.06 | 7.62 | |
| IMP-SFTBackbone=Qwen3-14B, Decoding=greedy decoding2026.06 | 7.53 | |
| Non-counterfactual-GRPOBackbone=Qwen3-8B, Decoding=greedy decoding2026.06 | 7.49 | |
| PRAGREST-SFTBackbone=Qwen3-8B, Decoding=greedy decoding2026.06 | 7.46 | |
| Non-counterfactual-SFTBackbone=Qwen3-8B, Decoding=greedy decoding2026.06 | 7.33 | |
| InstructBackbone=Qwen3-8B, Decoding=greedy decoding2026.06 | 7.24 | |
| DEEP-LAYER-DPOBackbone=Qwen3-8B, Decoding=greedy decoding2026.06 | 7.24 | |
| IMP-SFTBackbone=Qwen3-8B, Decoding=greedy decoding2026.06 | 7.19 |