Discriminative Reasoning on RPEval Single, Explicit Memory
0.02Ignorance ScoreQwen2.5-7B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen2.5-7BBase Model=Qwen2.5-7B, Prompt Strategy=Vanilla2026.01 | 0.02 | 0.74 | 0.3 | 0.35 | |
| Qwen2.5-7B + ReminderBase Model=Qwen2.5-7B, Prompt Strategy=Reminder2026.01 | 0.06 | 0.84 | 0.24 | 0.38 | |
| GPT-5Base Model=GPT-5, Prompt Strategy=Vanilla2026.01 | 0.06 | 0.56 | 0.94 | 0.52 | |
| GPT-5 + ReminderBase Model=GPT-5, Prompt Strategy=Reminder2026.01 | 0.12 | 0.58 | 0.82 | 0.51 | |
| DeepSeek-V3Base Model=DeepSeek-V3, Prompt Strategy=Vanilla2026.01 | 0.22 | 0.72 | 0.82 | 0.59 | |
| GPT-4.1Base Model=GPT-4.1, Prompt Strategy=Vanilla2026.01 | 0.26 | 0.34 | 0.92 | 0.51 | |
| GPT-4.1 + ReminderBase Model=GPT-4.1, Prompt Strategy=Reminder2026.01 | 0.28 | 0.34 | 0.96 | 0.53 | |
| GPT-5 + CoTBase Model=GPT-5, Prompt Strategy=Chain-of-Thought2026.01 | 0.28 | 0.68 | 0.94 | 0.63 | |
| Qwen2.5-7B + CoTBase Model=Qwen2.5-7B, Prompt Strategy=Chain-of-Thought2026.01 | 0.33 | 0.71 | 0.63 | 0.51 | |
| DeepSeek-V3 + ReminderBase Model=DeepSeek-V3, Prompt Strategy=Reminder2026.01 | 0.38 | 0.78 | 0.82 | 0.66 | |
| DeepSeek-V3 + CoTBase Model=DeepSeek-V3, Prompt Strategy=Chain-of-Thought2026.01 | 0.42 | 0.7 | 0.9 | 0.67 | |
| GPT-4.1 + CoTBase Model=GPT-4.1, Prompt Strategy=Chain-of-Thought2026.01 | 0.46 | 0.36 | 1 | 0.61 | |
| GPT-5 + RP-ReasonerBase Model=GPT-5, Prompt Strategy=RP-Reasoner2026.01 | 0.5 | 0.84 | 0.94 | 0.76 | |
| Qwen2.5-7B + RP-ReasonerBase Model=Qwen2.5-7B, Prompt Strategy=RP-Reasoner2026.01 | 0.66 | 0.52 | 0.5 | 0.56 | |
| DeepSeek-V3 + RP-ReasonerBase Model=DeepSeek-V3, Prompt Strategy=RP-Reasoner2026.01 | 0.7 | 0.7 | 0.78 | 0.73 | |
| GPT-4.1 + RP-ReasonerBase Model=GPT-4.1, Prompt Strategy=RP-Reasoner2026.01 | 0.7 | 0.7 | 0.9 | 0.77 |