Proactive Assistance on PROACTIVEBENCH (test)
100RecallGPT-4o-mini
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GPT-4o-miniEvaluation Protocol=Automatic Evaluation (LLM-as-Judge)2026.02 | 100 | 35.28 | 36.12 | 64.73 | 52.15 | |
| Qwen2-7B-ProactiveEvaluation Protocol=Automatic Evaluation (LLM-as-Judge)2026.02 | 100 | 49.78 | 50.66 | 50.22 | 66.47 | |
| Qwen3-8B-PRISMEvaluation Protocol=Human Expert Evaluation2026.02 | 99.41 | 74.01 | 73.79 | 25.91 | 84.85 | |
| LLaMA-3.1-8B-ProactiveEvaluation Protocol=Automatic Evaluation (LLM-as-Judge)2026.02 | 99.06 | 49.76 | 52.86 | 50.24 | 66.25 | |
| DeepSeek-R1Evaluation Protocol=Human Expert Evaluation2026.02 | 99.05 | 70.03 | 71.12 | 29.7 | 82.05 | |
| Qwen3-8B-PRISMEvaluation Protocol=Automatic Evaluation (LLM-as-Judge)2026.02 | 98.88 | 77.05 | 76.39 | 22.94 | 86.61 | |
| Deepseek-R1Evaluation Protocol=Automatic Evaluation (LLM-as-Judge)2026.02 | 98.12 | 72.35 | 72.96 | 27.64 | 83.28 | |
| GPT-4oEvaluation Protocol=Automatic Evaluation (LLM-as-Judge)2026.02 | 98.11 | 48.15 | 49.78 | 51.85 | 64.6 | |
| Claude-3.5-SonnetEvaluation Protocol=Automatic Evaluation (LLM-as-Judge)2026.02 | 97.89 | 45.37 | 49.78 | 54.63 | 62 | |
| Qwen3-8BEvaluation Protocol=Automatic Evaluation (LLM-as-Judge)2026.02 | 73.79 | 73.33 | 67.85 | 26.67 | 73.34 |