Generative Task Evaluation on Generative tasks single-preference setting
78Accuracy (Ignored)Qwen2.5-7B + RP-Reasoner
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Qwen2.5-7B + RP-ReasonerBase Model=Qwen2.5-7B, Prompting Baseline=RP-Reasoner2026.01 | 78 | 78 | 62 | 73 | 0.66 | 0.76 | 1.62 | 1.01 | |
| DeepSeek-V3 + RP-ReasonerBase Model=DeepSeek-V3, Prompting Baseline=RP-Reasoner2026.01 | 72 | 94 | 98 | 88 | 1.06 | 0.32 | 0.28 | 0.55 | |
| GPT-5 + RP-ReasonerBase Model=GPT-5, Prompting Baseline=RP-Reasoner2026.01 | 72 | 100 | 100 | 91 | 1.52 | 0.98 | 1.06 | 1.19 | |
| GPT-4.1 + RP-ReasonerBase Model=GPT-4.1, Prompting Baseline=RP-Reasoner2026.01 | 70 | 98 | 100 | 89 | 1.1 | 0.96 | 0.9 | 0.99 | |
| Qwen2.5-7BBase Model=Qwen2.5-7B, Prompting Baseline=Direct2026.01 | 46 | 94 | 60 | 67 | 1.94 | 1.08 | 1.96 | 1.66 | |
| Qwen2.5-7B + ReminderBase Model=Qwen2.5-7B, Prompting Baseline=Reminder2026.01 | 38 | 96 | 66 | 67 | 2.2 | 1.04 | 1.72 | 1.65 | |
| GPT-5 + ReminderBase Model=GPT-5, Prompting Baseline=Reminder2026.01 | 8 | 100 | 100 | 69 | 3.08 | 1 | 1.04 | 1.71 | |
| GPT-5Base Model=GPT-5, Prompting Baseline=Direct2026.01 | 7 | 100 | 100 | 61 | 3.22 | 1.02 | 1.07 | 1.79 | |
| GPT-4.1Base Model=GPT-4.1, Prompting Baseline=Direct2026.01 | 6 | 100 | 100 | 69 | 3.28 | 0.9 | 0.98 | 1.72 | |
| DeepSeek-V3 + ReminderBase Model=DeepSeek-V3, Prompting Baseline=Reminder2026.01 | 4 | 98 | 98 | 67 | 3.72 | 1.06 | 0.92 | 1.9 | |
| Qwen2.5-7B + CoTBase Model=Qwen2.5-7B, Prompting Baseline=Chain-of-Thought2026.01 | 2 | 98 | 88 | 63 | 3.74 | 1.2 | 1.74 | 2.23 | |
| DeepSeek-V3Base Model=DeepSeek-V3, Prompting Baseline=Direct2026.01 | 2 | 100 | 96 | 66 | 3.98 | 0.98 | 0.96 | 1.97 | |
| GPT-4.1 + ReminderBase Model=GPT-4.1, Prompting Baseline=Reminder2026.01 | 2 | 100 | 100 | 67 | 3.24 | 1 | 0.94 | 1.73 | |
| DeepSeek-V3 + CoTBase Model=DeepSeek-V3, Prompting Baseline=Chain-of-Thought2026.01 | 0 | 100 | 100 | 67 | 3.84 | 1.1 | 1.64 | 2.19 | |
| GPT-4.1 + CoTBase Model=GPT-4.1, Prompting Baseline=Chain-of-Thought2026.01 | 0 | 100 | 98 | 66 | 3.9 | 1.24 | 1.52 | 2.22 | |
| GPT-5 + CoTBase Model=GPT-5, Prompting Baseline=Chain-of-Thought2026.01 | 0 | 100 | 100 | 67 | 3.88 | 1.12 | 1.5 | 2.16 |