Mathematical Reasoning on CHAMP standard (test)
68.5AccuracyMDToC
Evaluation Results
| Method | Links | |
|---|---|---|
| MDToCLLM=GPT-4o, Prompt=MDToC2025.12 | 68.5 | |
| GoTLLM=GPT-4o, Prompt=GoT2025.12 | 63.4 | |
| ToTLLM=GPT-4o, Prompt=ToT2025.12 | 61.3 | |
| MDToCLLM=GPT-4-Turbo, Prompt=MDToC2025.12 | 59 | |
| GoTLLM=GPT-4-Turbo, Prompt=GoT2025.12 | 54.2 | |
| ToTLLM=GPT-4-Turbo, Prompt=ToT2025.12 | 52.7 | |
| MDToCLLM=GPT-4o-mini, Prompt=MDToC2025.12 | 47.1 | |
| Ours (theory-guided context selection strategy)Model=Qwen3-8B2026.02 | 41.1 | |
| ExpRAGModel=Qwen3-8B2026.02 | 40.7 | |
| ReMemModel=Qwen3-8B2026.02 | 40.7 | |
| DCModel=Qwen3-8B2026.02 | 40.4 | |
| BM25Model=Qwen3-8B2026.02 | 40 | |
| ZeroModel=Qwen3-8B2026.02 | 39.6 | |
| GoTLLM=GPT-4o-mini, Prompt=GoT2025.12 | 39.3 | |
| ToTLLM=GPT-4o-mini, Prompt=ToT2025.12 | 37.8 | |
| MDToCLLM=Llama-3-70B, Prompt=MDToC2025.12 | 36.1 | |
| MDToCLLM=Mistral-8x22B, Prompt=MDToC2025.12 | 34.5 | |
| MDToCLLM=GPT-3.5-Turbo, Prompt=MDToC2025.12 | 33.9 | |
| GoTLLM=Llama-3-70B, Prompt=GoT2025.12 | 32.7 | |
| GoTLLM=Mistral-8x22B, Prompt=GoT2025.12 | 32.4 | |
| GoTLLM=GPT-3.5-Turbo, Prompt=GoT2025.12 | 32.3 | |
| ToTLLM=Mistral-8x22B, Prompt=ToT2025.12 | 31.9 | |
| ToTLLM=GPT-3.5-Turbo, Prompt=ToT2025.12 | 31.7 | |
| ToTLLM=Llama-3-70B, Prompt=ToT2025.12 | 31.3 | |
| MDToCLLM=Llama-3-8B, Prompt=MDToC2025.12 | 21.5 | |
| GoTLLM=Llama-3-8B, Prompt=GoT2025.12 | 20.9 | |
| MDToCLLM=Mistral-7B, Prompt=MDToC2025.12 | 19.7 | |
| ToTLLM=Llama-3-8B, Prompt=ToT2025.12 | 19.2 | |
| GoTLLM=Mistral-7B, Prompt=GoT2025.12 | 18.8 | |
| ToTLLM=Mistral-7B, Prompt=ToT2025.12 | 18.1 | |
| ReMemModel=Llama-3.1-8B2026.02 | 15.2 | |
| Ours (theory-guided context selection strategy)Model=Llama-3.1-8B2026.02 | 15.2 | |
| DCModel=Llama-3.1-8B2026.02 | 14.8 | |
| ExpRAGModel=Llama-3.1-8B2026.02 | 14.8 | |
| BM25Model=Llama-3.1-8B2026.02 | 14.4 | |
| ZeroModel=Llama-3.1-8B2026.02 | 14.1 |