Multi-hop Question Answering on MuSiQue (Rubric Judge Accuracy)
39.6Rubric Judge AccuracyPredicate-Based Belief State
Evaluation results
| Method | Links | |
|---|---|---|
| Predicate-Based Belief StateHarness=Iter-RetGen, Representation=Freeform (b_free)2026.05 | 39.6 | |
| BaselineHarness=Iter-RetGen2026.05 | 37.1 | |
| Predicate-Based Belief StateHarness=ReAct, Representation=Freeform (b_free)2026.05 | 36.8 | |
| Predicate-Based Belief StateHarness=MemGPT, Representation=Freeform (b_free)2026.05 | 34.4 | |
| Predicate-Based Belief StateHarness=Iter-RetGen, Representation=Structured JSON (b_struct)2026.05 | 33.8 | |
| BaselineHarness=MemGPT2026.05 | 31.4 | |
| LobotomizedHarness=MemGPT2026.05 | 30.6 | |
| Predicate-Based Belief StateHarness=ReAct, Representation=Structured JSON (b_struct)2026.05 | 30.1 | |
| BaselineHarness=ReAct2026.05 | 29.6 | |
| Predicate-Based Belief StateHarness=MemGPT, Representation=Structured JSON (b_struct)2026.05 | 29.6 | |
| Predicate-Based Belief StateHarness=IRCoT, Representation=Freeform (b_free)2026.05 | 25.9 | |
| Predicate-Based Belief StateHarness=IRCoT, Representation=Structured JSON (b_struct)2026.05 | 21.1 | |
| BaselineHarness=IRCoT2026.05 | 14.4 | |
| LobotomizedHarness=ReAct2026.05 | 12.2 | |
| LobotomizedHarness=IRCoT2026.05 | 9 |