Theorem-based Reasoning on TheoremQA
53ScorerSIM
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| rSIMReasoner=7B, Planner=7B2025.12 | 53 | 5 | |
| rSIMReasoner=7B, Planner=0.5B2025.12 | 48.3 | 6 | |
| PromptingReasoner=14B, Planner=14B2025.12 | 47.4 | 5 | |
| PromptingReasoner=14B, Planner=7B2025.12 | 43.8 | 7 | |
| PS+Reasoner=14B, Planner=No2025.12 | 43.4 | 0 | |
| ZeroCoTReasoner=14B, Planner=No2025.12 | 43 | 0 | |
| rSIM (Plug-in)Reasoner=Llama3.3-70B, Planner=3B plug-in2025.12 | 41.8 | 6 | |
| PromptingReasoner=7B, Planner=14B2025.12 | 41.8 | 5 | |
| rSIM (Transfer)Reasoner=Llama3.3-70B, Planner=Qwen2.5-1.5B2025.12 | 40.7 | 6 | |
| rSIM (Plug-in)Reasoner=Llama3.3-70B, Planner=1B plug-in2025.12 | 39 | 5 | |
| rSIMReasoner=0.5B, Planner=7B2025.12 | 38.7 | 4 | |
| PromptingReasoner=Llama3.3-70B, Planner=70B2025.12 | 38.6 | 5 | |
| PromptingReasoner=7B, Planner=7B2025.12 | 36.4 | 7 | |
| ZeroCoTReasoner=7B, Planner=No2025.12 | 36 | 0 | |
| PSReasoner=7B, Planner=No2025.12 | 34.9 | 0 | |
| rSIMReasoner=0.5B, Planner=0.5B2025.12 | 34.1 | 3 | |
| ZeroCoTReasoner=Llama3.3-70B, Planner=No2025.12 | 32.3 | 0 | |
| PS+Reasoner=Llama3.3-70B, Planner=No2025.12 | 32 | 0 | |
| PromptingReasoner=Llama3.3-70B, Planner=3B2025.12 | 31.9 | 8 | |
| rSIMReasoner=Llama3.2-1B, Planner=3B2025.12 | 25 | 6 | |
| rSIMReasoner=Llama3.2-1B, Planner=Qwen2.5-1.5B2025.12 | 24.2 | 5 | |
| PromptingReasoner=Llama3.2-3B, Planner=70B2025.12 | 22.8 | 6 | |
| rSIMReasoner=Llama3.2-1B, Planner=1B2025.12 | 20.9 | 3 | |
| ZeroCoTReasoner=Llama3.2-3B, Planner=No2025.12 | 20.7 | 0 | |
| PromptingReasoner=Llama3.2-3B, Planner=3B2025.12 | 19.9 | 8 | |
| PS+Reasoner=Llama3.2-3B, Planner=No2025.12 | 18.6 | 0 | |
| PromptingReasoner=Llama3.2-1B, Planner=70B2025.12 | 14.3 | 5 | |
| ZeroCoTReasoner=Llama3.2-1B, Planner=No2025.12 | 13.7 | 0 | |
| PS+Reasoner=Llama3.2-1B, Planner=No2025.12 | 12.2 | 0 | |
| PromptingReasoner=0.5B, Planner=14B2025.12 | 10 | 5 | |
| PromptingReasoner=0.5B, Planner=7B2025.12 | 9.6 | 8 | |
| ZeroCoTReasoner=0.5B, Planner=No2025.12 | 9.5 | 0 | |
| PS+Reasoner=0.5B, Planner=No2025.12 | 8 | 0 | |
| PromptingReasoner=Llama3.2-1B, Planner=3B2025.12 | 6.6 | 6 |