Active Reasoning on AR-Bench-DC
61Exact AccuracyBALAR
Evaluation results
| Method | Links | |
|---|---|---|
| BALARLLM Backbone=Llama-3.1-8B-Instruct, Max ask rounds=252026.05 | 61 | |
| ToTLLM Backbone=Llama-3.1-8B-Instruct, Max ask rounds=252026.05 | 54 | |
| Proactive CoTLLM Backbone=Llama-3.1-8B-Instruct, Max ask rounds=252026.05 | 49 | |
| BALARLLM Backbone=Qwen2.5-32B-Instruct, Max ask rounds=252026.05 | 47 | |
| o4-miniSize=–2026.05 | 46.7 | |
| TeamTR (Homogeneous)Size=3×8B2026.05 | 46.7 | |
| TeamTR (Heterogeneous)Size=1.7B+8B+14B2026.05 | 45.5 | |
| Debate + JudgeSize=3×8B2026.05 | 44.3 | |
| GPT-4o-miniSize=–2026.05 | 44 | |
| Role-playSize=3×8B2026.05 | 43.7 | |
| Qwen3-A3BSize=30B2026.05 | 43.1 | |
| DebateSize=3×8B2026.05 | 43.1 | |
| Few-ShotLLM Backbone=Llama-3.1-8B-Instruct, Max ask rounds=252026.05 | 43 | |
| Qwen3 (thinking)Size=32B2026.05 | 42.7 | |
| DeepSeek-R1 DistillSize=70B2026.05 | 42.1 | |
| QwQSize=32B2026.05 | 41.9 | |
| Few-ShotLLM Backbone=Qwen2.5-32B-Instruct, Max ask rounds=252026.05 | 41 | |
| DAPOSize=8B2026.05 | 40.7 | |
| GRPOSize=8B2026.05 | 40.1 | |
| UoTLLM Backbone=Llama-3.1-8B-Instruct, Max ask rounds=252026.05 | 37 | |
| UoTLLM Backbone=Qwen2.5-32B-Instruct, Max ask rounds=252026.05 | 15 | |
| Proactive CoTLLM Backbone=Qwen2.5-32B-Instruct, Max ask rounds=252026.05 | 8 | |
| ToTLLM Backbone=Qwen2.5-32B-Instruct, Max ask rounds=252026.05 | 5 |