List your company
ResearchFormerly WizWand

Pragmatic Reasoning on ALTPRAG

8.14Reference-Based ScorePRAGREST-GRPO
7.1527.40857.6657.9215Jun 17, 2026
Updated 4mo ago

Evaluation results

MethodLinks
PRAGREST-GRPOBackbone=Qwen3-14B, Decoding=greedy decoding2026.068.14
PRAGREST-SFTBackbone=Qwen3-14B, Decoding=greedy decoding2026.068.06
Non-counterfactual-GRPOBackbone=Qwen3-14B, Decoding=greedy decoding2026.067.87
Non-counterfactual-SFTBackbone=Qwen3-14B, Decoding=greedy decoding2026.067.8
InstructBackbone=Qwen3-14B, Decoding=greedy decoding2026.067.78
DEEP-LAYER-DPOBackbone=Qwen3-14B, Decoding=greedy decoding2026.067.71
PRAGREST-GRPOBackbone=Qwen3-8B, Decoding=greedy decoding2026.067.62
IMP-SFTBackbone=Qwen3-14B, Decoding=greedy decoding2026.067.53
Non-counterfactual-GRPOBackbone=Qwen3-8B, Decoding=greedy decoding2026.067.49
PRAGREST-SFTBackbone=Qwen3-8B, Decoding=greedy decoding2026.067.46
Non-counterfactual-SFTBackbone=Qwen3-8B, Decoding=greedy decoding2026.067.33
InstructBackbone=Qwen3-8B, Decoding=greedy decoding2026.067.24
DEEP-LAYER-DPOBackbone=Qwen3-8B, Decoding=greedy decoding2026.067.24
IMP-SFTBackbone=Qwen3-8B, Decoding=greedy decoding2026.067.19