Tutor Agent Evaluation on DICT
74RecallOracle
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| OracleBackbone Model=–2025.02 | 74 | 61.2 | 51.9 | 144.8 | 200.8 | 318.5 | 42.3 | 176.1 | 21.1 | 60.7 | |
| TRAVERBackbone Model=GPT-4o2025.02 | 68.8 | 49.8 | 43.7 | 106.5 | 166.3 | 242.5 | 34.8 | 122.9 | 15.8 | 44.8 | |
| TRAVERBackbone Model=Llama-3.1-70B-Instruct2025.02 | 66.8 | 45.5 | 39.3 | 85.7 | 164.5 | 206.9 | 23.4 | 104.1 | 16.4 | 28.4 | |
| TreeInstructBackbone Model=GPT-4o2025.02 | 64.3 | 40.1 | 39.8 | 88.1 | 154.8 | 211.5 | 21.8 | 100.3 | 9.7 | 33.6 | |
| Vanilla InstructBackbone Model=GPT-4o2025.02 | 64.2 | 39.9 | 38.7 | 82.8 | 141.3 | 207.8 | 28.4 | 102.1 | 8.9 | 23.5 | |
| Self-RefineBackbone Model=GPT-4o2025.02 | 64 | 39.5 | 40.6 | 91.7 | 143 | 221.4 | 23.1 | 118.6 | 11.8 | 26.2 | |
| Vanilla InstructBackbone Model=Llama-3.1-8B-Instruct2025.02 | 63.6 | 38.5 | 31.1 | 46.7 | 138.6 | 145.1 | 23.8 | 48.3 | 10.9 | 8 | |
| Vanilla InstructBackbone Model=Llama-3.1-70B-Instruct2025.02 | 62.5 | 36 | 34.9 | 65 | 127.4 | 160.6 | 21.5 | 71 | 11.7 | 24.9 | |
| Vanilla InstructBackbone Model=Qwen2-72B-Instruct2025.02 | 61.4 | 33.8 | 32.1 | 51.4 | 131.8 | 128.2 | 13.7 | 53.7 | 11.5 | 21.4 | |
| Vanilla InstructBackbone Model=o1-mini2025.02 | 61.3 | 33.4 | 35.9 | 69.4 | 129.5 | 159.1 | 19.3 | 101.2 | 6.9 | 16.6 | |
| Vanilla InstructBackbone Model=GPT-3.5-Turbo2025.02 | 60.1 | 31 | 28.8 | 35.9 | 110.9 | 130.1 | 21.6 | 37.7 | 6.9 | -1.3 | |
| Vanilla InstructBackbone Model=Qwen2-7B-Instruct2025.02 | 56.4 | 22.8 | 26.4 | 24.5 | 99.4 | 69.8 | 10.1 | 37.5 | 3 | -0.3 | |
| Pre-TestBackbone Model=–2025.02 | 45.9 | — | 21.2 | — | — | — | — | — | — | — |