Proactive task scheduling on Home Loan (test)
73.03PRIGemini-2.5-flash
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Gemini-2.5-flashReasoning=true, ASG=false2026.05 | 73.03 | 34.5 | 52.7 | 52.8 | 0.0757 | 0.01 | 24.1 | |
| Claude-4Reasoning=true, ASG=true2026.05 | 72.62 | 37.5 | 60.7 | 61.9 | 0.076 | 1.27 | 30.7 | |
| Claude-4Reasoning=true, ASG=false2026.05 | 70.39 | 33.2 | 47.2 | 42.2 | 0.0742 | 0.63 | 26.1 | |
| Qwen2.5-14B-ProActor-Q4RULER=Adaptive2026.05 | 62.32 | 39.5 | 46.6 | 18 | 0.0501 | 1.31 | 15.6 | |
| Gemini-2.5-flashReasoning=false, ASG=false2026.05 | 61.65 | 34.9 | 68.8 | 97.2 | 0.0632 | 0.72 | 17.3 | |
| Gemini-2.5-flashReasoning=true, ASG=true2026.05 | 60.67 | 28.3 | 47.9 | 69.3 | 0.0764 | 0.63 | 25.1 | |
| Qwen2.5-14B-ProActor-Q4RULER=Custom2026.05 | 56.03 | 20.6 | 23.4 | 13.7 | 0.0846 | 3.55 | 46.5 | |
| Claude-4Reasoning=false, ASG=false2026.05 | 54.16 | 22.4 | 34.7 | 54.9 | 0.0811 | 1.18 | 33.2 | |
| GPT-5.1Reasoning=true, ASG=true2026.05 | 51.01 | 28.1 | 48.1 | 71.2 | 0.0374 | 0.34 | 10.5 | |
| GPT-5.1Reasoning=false, ASG=false2026.05 | 50.47 | 27.2 | 46.7 | 71.7 | 0.0462 | 0.49 | 11.6 | |
| GPT-5.1Reasoning=true, ASG=false2026.05 | 50.47 | 36.3 | 57.9 | 59.5 | 0.0186 | 0.03 | 3.1 | |
| Qwen2.5-14B-InstructReasoning=false, ASG=false2026.05 | 42.88 | 25.3 | 38.3 | 51.4 | 0.012 | 0.03 | 5.4 | |
| Qwen2.5-14B-InstructReasoning=true, ASG=false2026.05 | 40.12 | 21.7 | 23.9 | 10.1 | 0.0037 | 0.06 | 3.2 | |
| Qwen2.5-14B-InstructReasoning=true, ASG=true2026.05 | 32.23 | 11.3 | 14.4 | 27.4 | 0.0184 | 0.33 | 9 |