Discriminative Task on RPEval Implicit Memory, Multi-Preference
0.44IA (Macro)GPT-4.1 + RP-Reasoner
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GPT-4.1 + RP-ReasonerPrompting Strategy=RP-Reasoner2026.01 | 0.44 | 0.08 | 0.22 | 0.67 | 0.55 | 0.59 | |
| DeepSeek-V3 + CoTPrompting Strategy=Chain-of-Thought2026.01 | 0.38 | 0.13 | 0.23 | 0.7 | 0.61 | 0.64 | |
| Qwen2.5-7B + RP-ReasonerPrompting Strategy=RP-Reasoner2026.01 | 0.31 | 0.02 | 0.13 | 0.52 | 0.44 | 0.46 | |
| DeepSeek-V3 + RP-ReasonerPrompting Strategy=RP-Reasoner2026.01 | 0.31 | 0.14 | 0.21 | 0.65 | 0.65 | 0.65 | |
| GPT-5 + CoTPrompting Strategy=Chain-of-Thought2026.01 | 0.26 | 0.06 | 0.15 | 0.46 | 0.49 | 0.48 | |
| GPT-5 + RP-ReasonerPrompting Strategy=RP-Reasoner2026.01 | 0.19 | 0.09 | 0.13 | 0.53 | 0.54 | 0.53 | |
| Qwen2.5-7BPrompting Strategy=Vanilla2026.01 | 0.15 | 0 | 0.06 | 0.36 | 0.22 | 0.27 | |
| DeepSeek-V3 + ReminderPrompting Strategy=Reminder2026.01 | 0.14 | 0.07 | 0.09 | 0.62 | 0.55 | 0.57 | |
| GPT-4.1 + CoTPrompting Strategy=Chain-of-Thought2026.01 | 0.14 | 0.08 | 0.1 | 0.38 | 0.49 | 0.46 | |
| DeepSeek-V3Prompting Strategy=Vanilla2026.01 | 0.12 | 0.07 | 0.09 | 0.6 | 0.56 | 0.57 | |
| Qwen2.5-7B + CoTPrompting Strategy=Chain-of-Thought2026.01 | 0.05 | 0 | 0.02 | 0.42 | 0.32 | 0.35 | |
| GPT-4.1 + ReminderPrompting Strategy=Reminder2026.01 | 0.05 | 0.01 | 0.03 | 0.31 | 0.231 | 0.26 | |
| GPT-5Prompting Strategy=Vanilla2026.01 | 0.05 | 0.01 | 0.03 | 0.32 | 0.42 | 0.39 | |
| Qwen2.5-7B + ReminderPrompting Strategy=Reminder2026.01 | 0.03 | 0.01 | 0.02 | 0.27 | 0.26 | 0.26 | |
| GPT-5 + ReminderPrompting Strategy=Reminder2026.01 | 0.03 | 0.04 | 0.03 | 0.32 | 0.43 | 0.39 | |
| GPT-4.1Prompting Strategy=Vanilla2026.01 | 0.02 | 0 | 0.01 | 0.27 | 0.19 | 0.22 |