Question Answering on Multi-objective QA (Varying Objectives)
47.7Score (2 Objectives)ECHO w/ Top-K retrieval
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| ECHO w/ Top-K retrievalBackbone=Qwen3-32B-Instruct, Zero-shot protocol=true, Retrieval Mechanism=Top-K retrieval2026.06 | 47.7 | 39.2 | 27.8 | — | 42 | 39.2 | |
| ECHOBackbone=Qwen3-32B-Instruct, Zero-shot protocol=true2026.06 | 47.7 | 41.5 | 36.1 | — | 45.5 | 42.7 | |
| ECHO w/o traceable CABackbone=Qwen3-32B-Instruct, Zero-shot protocol=true, Credit Assignment (CA)=w/o traceable CA2026.06 | 45.5 | 39.2 | — | — | 42 | 37.4 | |
| ECHO w/o Traceable CA & Turn-level ISBackbone=Qwen3-32B-Instruct, Zero-shot protocol=true, Credit Assignment (CA)=w/o Traceable CA, Turn-level Importance Sampling (IS)=w/o Turn-level IS2026.06 | 45.5 | 35.2 | 27 | — | 47.7 | 38.8 | |
| SUPOBackbone=Qwen3-32B-Instruct, Zero-shot protocol=true2026.06 | 40.9 | 36.4 | 34.7 | — | 36.4 | 37.1 | |
| ECHO w/ Top-K retrieval & w/o turn summaryBackbone=Qwen3-32B-Instruct, Zero-shot protocol=true, Retrieval Mechanism=Top-K retrieval, Turn Summary Configuration=w/o turn summary2026.06 | 40.9 | 35.8 | 35.5 | — | 44.3 | 39.1 | |
| GRPOBackbone=Qwen3-32B-Instruct, Zero-shot protocol=true2026.06 | 38.6 | 35.8 | 29 | — | 39.8 | 35.8 | |
| ECHOBackbone=Qwen3-30B-A3B-Instruct, Zero-shot protocol=true2026.06 | 34.1 | 30.1 | 18.8 | — | 36.4 | 29.9 | |
| GRPOBackbone=Qwen3-30B-A3B-Instruct, Zero-shot protocol=true2026.06 | 27.3 | 27.3 | 16.2 | — | 26.1 | 24.2 | |
| SUPOBackbone=Qwen3-30B-A3B-Instruct, Zero-shot protocol=true2026.06 | 25 | 27.3 | — | — | 30.7 | 25.3 | |
| BACM-RLBackbone=Qwen3-30B-A3B-Inst, Context=8k2026.04 | 1.032 | 3.587 | 6.255 | 4.545 | — | — | |
| Search-R1(RL)Backbone=Qwen3-30B-A3B-Inst, Context=8k2026.04 | 1.013 | 3.31 | 1.949 | 0.998 | — | — | |
| MEM1Backbone=Qwen3-30B-A3B-Inst, Context=8k2026.04 | 0.978 | 1.327 | 1.383 | 0.909 | — | — | |
| ReACTBackbone=Qwen3-235B-A22B-Inst-2507, Context=128k2026.04 | 0.948 | 1.873 | 1.315 | 1.412 | — | — | |
| ReACTBackbone=Qwen3-30B-A3B-Inst, Context=8k2026.04 | 0.938 | 2.078 | 0.931 | 0.208 | — | — | |
| SummaryBackbone=Qwen3-30B-A3B-Inst, Context=8k2026.04 | 0.916 | 2.456 | 2.992 | 2.848 | — | — | |
| BACM-RLBackbone=Qwen2.5-7B-Inst, Context=8k2026.04 | 0.909 | 2.79 | — | 2.938 | — | — | |
| ReACTBackbone=Qwen3-235B-A22B-Inst-2507, Context=8k2026.04 | 0.886 | 1.782 | 1.233 | 0.374 | — | — | |
| MEM1(RL)Backbone=Qwen2.5-7B-Inst, Context=8k2026.04 | 0.838 | 2.345 | 2.391 | 1.21 | — | — | |
| Search-R1(RL)Backbone=Qwen2.5-7B-Inst, Context=8k2026.04 | 0.76 | 1.719 | 2.497 | 1.022 | — | — | |
| BACM (w/o RL)Backbone=Qwen2.5-7B-Inst, Context=8k2026.04 | 0.69 | 1.698 | 1.179 | 0.305 | — | — | |
| SummaryBackbone=Qwen2.5-7B-Inst, Context=8k2026.04 | 0.678 | 2.176 | 2.379 | 0.567 | — | — | |
| BACM (w/o RL)Backbone=Qwen3-30B-A3B-Inst, Context=8k2026.04 | 0.474 | 0.098 | 0.17 | 0.208 | — | — | |
| ReACTBackbone=Qwen2.5-7B-Inst, Context=8k2026.04 | 0.395 | 0.458 | 0.376 | 0.09 | — | — |