Online Auditing on AFTRAJ-2K
77.36Exact-F1 (Math)AgentForesight-7B
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| AgentForesight-7BBackbone=Qwen2.5-7B-Instruct2026.05 | 77.36 | 0.96 | 78.87 | 0.18 | 48.7 | 0.54 | 66.44 | 0.59 | |
| DeepSeek-V4-ProModel Category=Proprietary LLMs2026.05 | 50.34 | 2.6 | 49.32 | 0.96 | 41.77 | 1.31 | 46.56 | 1.77 | |
| DeepSeek-V4-FlashModel Category=Proprietary LLMs2026.05 | 42.77 | 2.78 | 38.1 | 1.27 | 32.53 | 1.42 | 37.65 | 1.94 | |
| Gemini-3-FlashModel Category=Proprietary LLMs2026.05 | 40.52 | 2.48 | 19.42 | 2.73 | 26.09 | 1.7 | 29.74 | 2.19 | |
| GPT-4.1Model Category=Proprietary LLMs2026.05 | 24.39 | 3.95 | 10.81 | 3.5 | 40.68 | 1.29 | 27.43 | 2.67 | |
| Qwen3-8BModel Category=Open-Source LLMs2026.05 | 21.95 | 4.65 | 27.85 | 2.57 | 33.64 | 1.57 | 28.36 | 2.65 | |
| ToT-7BBackbone=Qwen2.5-7B-Instruct2026.05 | 20.38 | 4.4 | 7.02 | 4.06 | 24.84 | 2.13 | 18.52 | 3.39 | |
| Claude-Haiku-4.5Model Category=Proprietary LLMs2026.05 | 19.75 | 4.12 | 23.91 | 2.8 | 31.95 | 1.69 | 25.53 | 2.85 | |
| Qwen3-32BModel Category=Open-Source LLMs2026.05 | 18.63 | 4.07 | 20 | 2.83 | 40 | 1.59 | 26.91 | 2.82 | |
| Reflexion-7BBackbone=Qwen2.5-7B-Instruct2026.05 | 16.57 | 4.39 | 9.52 | 4.5 | 39.13 | 1.44 | 23.38 | 3.17 | |
| Qwen2.5-7B-InstructModel Category=Open-Source LLMs2026.05 | 10.39 | 3.8 | 38.2 | 2.26 | 14 | 2.96 | 21.05 | 2.75 | |
| Llama3.2-3BModel Category=Open-Source LLMs2026.05 | 8.14 | 4.86 | 21.05 | 2.94 | 16.13 | 2.3 | 14.41 | 3.38 | |
| Perplexity-7BBackbone=Qwen2.5-7B-Instruct2026.05 | 2.31 | 4.09 | 26.56 | 2.19 | 16.57 | 2.5 | 14.11 | 3.02 | |
| Gemma3-4BModel Category=Open-Source LLMs2026.05 | 1.15 | 5.78 | 12.9 | 4.59 | 8.29 | 3.04 | 6.92 | 4.37 | |
| AgentDebug-7BBackbone=Qwen2.5-7B-Instruct2026.05 | 0 | — | 28.57 | 4.05 | 2.82 | 1 | 9.63 | 3.76 |