Report Generation on TMALL
4.6AccuracyRecPilot
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| RecPilotEvaluator=Simulated User2026.03 | 4.6 | 3.62 | 3.8 | 4.8 | 3.94 | 4.09 | 4.14 | |
| Plan-and-SolveEvaluator=Simulated User2026.03 | 4.48 | 3.52 | 3.76 | 4.55 | 3.9 | 3.07 | 3.88 | |
| DeepSeek-V3.2Evaluator=Simulated User2026.03 | 4.46 | 3.44 | 3.67 | 4.5 | 3.88 | 2.86 | 3.8 | |
| ReActEvaluator=Simulated User2026.03 | 4.45 | 3.35 | 3.62 | 4.41 | 3.82 | 2.92 | 3.76 | |
| GLM-5Evaluator=Simulated User2026.03 | 4.38 | 3.3 | 3.47 | 4.24 | 3.7 | 2.73 | 3.64 | |
| RecPilotEvaluator=Real User2026.03 | 4.36 | 3.6 | 3.82 | 4.69 | 4 | 4.06 | 4.09 | |
| GPT-5.2Evaluator=Simulated User2026.03 | 4.35 | 3.4 | 3.77 | 4.67 | 3.91 | 2.89 | 3.83 | |
| DeepSeek-V3.2Evaluator=Real User2026.03 | 4.25 | 3.42 | 3.79 | 4.35 | 4.04 | 2.95 | 3.8 | |
| Plan-and-SolveEvaluator=Real User2026.03 | 4.25 | 3.45 | 3.83 | 4.41 | 4 | 3.13 | 3.84 | |
| ReActEvaluator=Real User2026.03 | 4.17 | 3.26 | 3.65 | 4.2 | 3.98 | 2.95 | 3.7 | |
| GLM-5Evaluator=Real User2026.03 | 4.13 | 3.19 | 3.57 | 4.1 | 3.85 | 2.76 | 3.6 | |
| GPT-5.2Evaluator=Real User2026.03 | 4.07 | 3.41 | 3.83 | 4.54 | 4.05 | 2.97 | 3.81 | |
| Qwen3-MaxEvaluator=Simulated User2026.03 | 4.05 | 3.24 | 3.66 | 4.48 | 3.78 | 2.91 | 3.68 | |
| Qwen3-MaxEvaluator=Real User2026.03 | 3.76 | 3.29 | 3.77 | 4.35 | 3.94 | 2.94 | 3.67 |