Intervention Timing Classification on BSDD
67.4AccuracyPULI
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PULILLM Backbone (Observer + Presenter)=(LLaMA3.2-1B-Instruct + LLaMA3.1-8B-Instruct)2026.04 | 67.4 | 61.7 | 69.6 | 65.4 | |
| PULILLM Backbone (Observer + Presenter)=(Qwen3-0.6B + Qwen3-14B)2026.04 | 64.1 | 31.2 | 91 | 46.4 | |
| ICLLLM Backbone (Observer + Presenter)=(GPT-4o-mini + GPT-4o)2026.04 | 63.8 | 34.1 | 83.8 | 48.5 | |
| Proactive AgentLLM Backbone (Observer + Presenter)=(GPT-4o-mini + GPT-4o)2026.04 | 62.3 | 31.7 | 81.5 | 45.6 | |
| Vanilla SFTLLM Backbone (Observer + Presenter)=(LLaMA3.2-1B-Instruct + LLaMA3.1-8B-Instruct)2026.04 | 61.7 | 57.5 | 62.8 | 60 | |
| Vanilla SFTLLM Backbone (Observer + Presenter)=(Qwen3-0.6B + Qwen3-14B)2026.04 | 58.6 | 20.9 | 85.4 | 33.7 | |
| ICLLLM Backbone (Observer + Presenter)=(LLaMA3.2-1B-Instruct + LLaMA3.1-8B-Instruct)2026.04 | 58.4 | 54.5 | 59.1 | 56.7 | |
| ICLLLM Backbone (Observer + Presenter)=(Qwen3-0.6B + Qwen3-14B)2026.04 | 55.7 | 16.2 | 79.4 | 28.9 | |
| Proactive AgentLLM Backbone (Observer + Presenter)=(LLaMA3.2-1B-Instruct + LLaMA3.1-8B-Instruct)2026.04 | 54.5 | 53.4 | 68.8 | 60.2 | |
| Proactive AgentLLM Backbone (Observer + Presenter)=(Qwen3-0.6B + Qwen3-14B)2026.04 | 53.9 | 14.9 | 67.6 | 24.5 | |
| StandardLLM Backbone (Observer + Presenter)=(GPT-4o-mini + GPT-4o)2026.04 | 50 | 0 | 0 | 0 | |
| StandardLLM Backbone (Observer + Presenter)=(Qwen3-0.6B + Qwen3-14B)2026.04 | 50 | 0 | 0 | 0 | |
| StandardLLM Backbone (Observer + Presenter)=(LLaMA3.2-1B-Instruct + LLaMA3.1-8B-Instruct)2026.04 | 50 | 0 | 0 | 0 | |
| RandomLLM Backbone (Observer + Presenter)=(GPT-4o-mini + GPT-4o)2026.04 | 47.9 | 26.3 | 46.3 | 33.6 | |
| RandomLLM Backbone (Observer + Presenter)=(LLaMA3.2-1B-Instruct + LLaMA3.1-8B-Instruct)2026.04 | 47.9 | 39.5 | 47.5 | 43.1 | |
| RandomLLM Backbone (Observer + Presenter)=(Qwen3-0.6B + Qwen3-14B)2026.04 | 46.7 | 26.9 | 44.6 | 33.5 |