Simulated Decision-Making on GPT-4o-mini Test Set
68.6AccuracyGPT-4o-mini
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-4o-miniAssistant Model=none2026.04 | 68.6 | 0.275 | 0.372 | 0.261 | |
| Token Probs.Assistant Model=Qwen3-14B2026.04 | 68 | 0.2 | 0.35 | 0.238 | |
| OursAssistant Model=Qwen3-14B2026.04 | 67.3 | 0.155 | 0.283 | 0.219 | |
| Verbal Conf.Assistant Model=Qwen3-14B2026.04 | 67 | 0.302 | 0.495 | 0.281 | |
| Token Probs.Assistant Model=Qwen3-4B-Thinking-25072026.04 | 66.7 | 0.222 | 0.404 | 0.249 | |
| Token Probs.Assistant Model=Qwen3-8B2026.04 | 66.7 | 0.214 | 0.36 | 0.248 | |
| OursAssistant Model=Qwen3-4B-Thinking-25072026.04 | 66.3 | 0.119 | 0.4 | 0.209 | |
| Verbal Conf.Assistant Model=Qwen3-8B2026.04 | 66.2 | 0.315 | 0.456 | 0.292 | |
| OursAssistant Model=Qwen3-8B2026.04 | 66.2 | 0.16 | 0.238 | 0.22 | |
| Token Probs.Assistant Model=Assistant Average2026.04 | 65.9 | 0.211 | 0.397 | 0.247 | |
| OursAssistant Model=Assistant Average2026.04 | 65.5 | 0.126 | 0.245 | 0.218 | |
| Verbal Conf.Assistant Model=Qwen3-4B-Thinking-25072026.04 | 65.3 | 0.3 | 0.476 | 0.299 | |
| Verbal Conf.Assistant Model=Assistant Average2026.04 | 64.6 | 0.319 | 0.482 | 0.31 | |
| OursAssistant Model=Qwen3-1.7B2026.04 | 64.3 | 0.134 | 0.203 | 0.219 | |
| Token Probs.Assistant Model=Qwen3-0.6B2026.04 | 64.1 | 0.164 | 0.4 | 0.233 | |
| Token Probs.Assistant Model=Qwen3-1.7B2026.04 | 64 | 0.255 | 0.472 | 0.267 | |
| OursAssistant Model=Qwen3-0.6B2026.04 | 63.4 | 0.061 | 0.1 | 0.223 | |
| Verbal Conf.Assistant Model=Qwen3-1.7B2026.04 | 63.2 | 0.343 | 0.63 | 0.33 | |
| Verbal Conf.Assistant Model=Qwen3-0.6B2026.04 | 61.2 | 0.336 | 0.35 | 0.347 |