Question Answering, Reasoning, and Coding on ExpBench Generic
84ARC-C AccuracyExpWeaver
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ExpWeaverBackbone=Qwen2.5-3B-Instruct, Training=LoRA (rank 32), Algorithm=GRPO2026.05 | 84 | 82 | 35 | 68 | 80 | 89.78 | 85 | 65.33 | 72 | 78 | 78.25 | |
| Search-R1Method Category=LLM-Centric Experience Learning Baseline2026.05 | 80.44 | 76.67 | 23.33 | 67.78 | 73.33 | 85.11 | 77.56 | 59.78 | 71.79 | 67.5 | 73.27 | |
| Search-o1Method Category=LLM-Centric Experience Learning Baseline2026.05 | 80 | 72 | 28.33 | 63.78 | 71.56 | 66.89 | 63.11 | 30.89 | 54 | 57.02 | 63.1 | |
| HRPOMethod Category=General Reasoning-Based Baseline2026.05 | 79.33 | 75.33 | 20 | 67.11 | 72 | 88.22 | 80.89 | 63.11 | 74.36 | 70.89 | 74.04 | |
| R1Method Category=General Reasoning-Based Baseline2026.05 | 78.44 | 74.67 | 21.67 | 66.22 | 71.33 | 87.11 | 79.78 | 61.78 | 73.08 | 68.75 | 73.1 | |
| ExpeLMethod Category=Retrieval-Centric Experience Learning Baseline2026.05 | 76.44 | 66.44 | 26.67 | 67.54 | 69.56 | 61.33 | 58.89 | 36 | 60 | 67.54 | 60.94 | |
| ReasoningBankMethod Category=Retrieval-Centric Experience Learning Baseline2026.05 | 74.89 | 63.33 | 30 | 60.44 | 67.56 | 68 | 63.11 | 32.89 | 52 | 52.63 | 60.57 | |
| LLM-RMethod Category=Retrieval-Centric Experience Learning Baseline2026.05 | 74.89 | 67.56 | 16.67 | 61.33 | 71.78 | 73.78 | 65.11 | 35.33 | 64 | 67.54 | 63.52 | |
| IRCoTMethod Category=LLM-Centric Experience Learning Baseline2026.05 | 73.56 | 64.22 | 33.33 | 60.22 | 68.89 | 65.11 | 63.11 | 28.67 | 52 | 55.26 | 59.82 | |
| CoTMethod Category=General Reasoning-Based Baseline2026.05 | 62.44 | 56.44 | 26.67 | 58.22 | 64.22 | 71.11 | 66.44 | 53.11 | 66 | 72.81 | 61.4 |