Long-context Question Answering on LoCoMo (Rubric Judge)
67.3Rubric Judge AccuracyBaseline
Evaluation results
| Method | Links | |
|---|---|---|
| BaselineHarness=Iter-RetGen2026.05 | 67.3 | |
| Predicate-Based Belief StateHarness=ReAct, Representation=Freeform (b_free)2026.05 | 66.6 | |
| Predicate-Based Belief StateHarness=Iter-RetGen, Representation=Freeform (b_free)2026.05 | 65.9 | |
| Predicate-Based Belief StateHarness=ReAct, Representation=Structured JSON (b_struct)2026.05 | 65.7 | |
| Predicate-Based Belief StateHarness=IRCoT, Representation=Freeform (b_free)2026.05 | 65.4 | |
| Predicate-Based Belief StateHarness=MemGPT, Representation=Structured JSON (b_struct)2026.05 | 65.4 | |
| BaselineHarness=ReAct2026.05 | 65.1 | |
| Predicate-Based Belief StateHarness=Iter-RetGen, Representation=Structured JSON (b_struct)2026.05 | 65 | |
| Predicate-Based Belief StateHarness=IRCoT, Representation=Structured JSON (b_struct)2026.05 | 63.6 | |
| BaselineHarness=IRCoT2026.05 | 63 | |
| Predicate-Based Belief StateHarness=MemGPT, Representation=Freeform (b_free)2026.05 | 62.5 | |
| LobotomizedHarness=ReAct2026.05 | 59.4 | |
| LobotomizedHarness=IRCoT2026.05 | 57.2 | |
| BaselineHarness=MemGPT2026.05 | 57.2 | |
| LobotomizedHarness=MemGPT2026.05 | 56.5 |