Discriminative Reasoning on RPEval Multi-MACRO, Explicit Memory
0.4IA ScoreQwen2.5-7B + RP-Reasoner
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Qwen2.5-7B + RP-ReasonerBase Model=Qwen2.5-7B, Prompt Strategy=RP-Reasoner2026.01 | 0.4 | 0.01 | 0.17 | |
| DeepSeek-V3 + CoTBase Model=DeepSeek-V3, Prompt Strategy=Chain-of-Thought2026.01 | 0.4 | 0.1 | 0.23 | |
| GPT-4.1 + RP-ReasonerBase Model=GPT-4.1, Prompt Strategy=RP-Reasoner2026.01 | 0.38 | 0.2 | 0.27 | |
| GPT-5 + RP-ReasonerBase Model=GPT-5, Prompt Strategy=RP-Reasoner2026.01 | 0.38 | 0.23 | 0.3 | |
| DeepSeek-V3 + RP-ReasonerBase Model=DeepSeek-V3, Prompt Strategy=RP-Reasoner2026.01 | 0.35 | 0.29 | 0.31 | |
| GPT-4.1 + CoTBase Model=GPT-4.1, Prompt Strategy=Chain-of-Thought2026.01 | 0.2 | 0.09 | 0.13 | |
| Qwen2.5-7BBase Model=Qwen2.5-7B, Prompt Strategy=Vanilla2026.01 | 0.18 | 0.01 | 0.08 | |
| Qwen2.5-7B + ReminderBase Model=Qwen2.5-7B, Prompt Strategy=Reminder2026.01 | 0.12 | 0.02 | 0.06 | |
| GPT-5 + CoTBase Model=GPT-5, Prompt Strategy=Chain-of-Thought2026.01 | 0.12 | 0.11 | 0.11 | |
| DeepSeek-V3Base Model=DeepSeek-V3, Prompt Strategy=Vanilla2026.01 | 0.08 | 0.04 | 0.06 | |
| GPT-4.1 + ReminderBase Model=GPT-4.1, Prompt Strategy=Reminder2026.01 | 0.08 | 0.04 | 0.06 | |
| DeepSeek-V3 + ReminderBase Model=DeepSeek-V3, Prompt Strategy=Reminder2026.01 | 0.05 | 0.07 | 0.06 | |
| GPT-4.1Base Model=GPT-4.1, Prompt Strategy=Vanilla2026.01 | 0.05 | 0.01 | 0.03 | |
| Qwen2.5-7B + CoTBase Model=Qwen2.5-7B, Prompt Strategy=Chain-of-Thought2026.01 | 0.04 | 0.03 | 0.03 | |
| GPT-5Base Model=GPT-5, Prompt Strategy=Vanilla2026.01 | 0 | 0.04 | 0.03 | |
| GPT-5 + ReminderBase Model=GPT-5, Prompt Strategy=Reminder2026.01 | 0 | 0.03 | 0.02 |