Generative tasks multi-preference setting
63Macro Acc (IA)GPT-4.1 + RP-Reasoner
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| GPT-4.1 + RP-ReasonerBackbone Model=GPT-4.1, Prompting Strategy=RP-Reasoner2026.01 | 63 | 1 | 26 | 59 | 59 | 59 | 1.43 | 2.63 | 2.15 | |
| GPT-5 + RP-ReasonerBackbone Model=GPT-5, Prompting Strategy=RP-Reasoner2026.01 | 55 | 4 | 25 | 59 | 60 | 60 | 1.88 | 2.27 | 2.11 | |
| Qwen2.5-7B + RP-ReasonerBackbone Model=Qwen2.5-7B, Prompting Strategy=RP-Reasoner2026.01 | 42 | 3 | 19 | 45 | 51 | 49 | 2.13 | 2.88 | 2.58 | |
| DeepSeek-V3 + RP-ReasonerBackbone Model=DeepSeek-V3, Prompting Strategy=RP-Reasoner2026.01 | 42 | 4 | 19 | 42 | 58 | 53 | 2.23 | 2.67 | 2.49 | |
| Qwen2.5-7BBackbone Model=Qwen2.5-7B, Prompting Strategy=Base2026.01 | 18 | 1 | 8 | 22 | 57 | 45 | 3.05 | 2.69 | 2.83 | |
| Qwen2.5-7B + ReminderBackbone Model=Qwen2.5-7B, Prompting Strategy=Reminder2026.01 | 8 | 2 | 5 | 10 | 52 | 38 | 3.52 | 2.72 | 3.04 | |
| Qwen2.5-7B + CoTBackbone Model=Qwen2.5-7B, Prompting Strategy=CoT2026.01 | 0 | 0 | 0 | 0 | 53 | 35 | 4.25 | 3.64 | 3.89 | |
| DeepSeek-V3Backbone Model=DeepSeek-V3, Prompting Strategy=Base2026.01 | 0 | 2 | 1 | 0 | 57 | 38 | 4.39 | 2.87 | 3.48 | |
| DeepSeek-V3 + ReminderBackbone Model=DeepSeek-V3, Prompting Strategy=Reminder2026.01 | 0 | 2 | 1 | 0 | 58 | 39 | 4.47 | 2.79 | 3.46 | |
| DeepSeek-V3 + CoTBackbone Model=DeepSeek-V3, Prompting Strategy=CoT2026.01 | 0 | 3 | 2 | 0 | 57 | 38 | 4.12 | 3.09 | 3.52 | |
| GPT-4.1Backbone Model=GPT-4.1, Prompting Strategy=Base2026.01 | 0 | 1 | 1 | 2 | 59 | 40 | 4.13 | 2.6 | 3.21 | |
| GPT-4.1 + ReminderBackbone Model=GPT-4.1, Prompting Strategy=Reminder2026.01 | 0 | 1 | 1 | 2 | 56 | 38 | 4.05 | 2.84 | 3.33 | |
| GPT-4.1 + CoTBackbone Model=GPT-4.1, Prompting Strategy=CoT2026.01 | 0 | 1 | 1 | 0 | 62 | 42 | 4.23 | 3.19 | 3.61 | |
| GPT-5Backbone Model=GPT-5, Prompting Strategy=Base2026.01 | 0 | 1 | 1 | 3 | 59 | 40 | 4.1 | 2.76 | 3.3 | |
| GPT-5 + ReminderBackbone Model=GPT-5, Prompting Strategy=Reminder2026.01 | 0 | 2 | 1 | 2 | 60 | 39 | 4.03 | 2.63 | 3.23 | |
| GPT-5 + CoTBackbone Model=GPT-5, Prompting Strategy=CoT2026.01 | 0 | 1 | 1 | 0 | 60 | 37 | 4.26 | 3.31 | 3.73 |