# Question Answering on LoCoMo 1.0 (test)

Source: [Question Answering on LoCoMo 1.0 (test)](<https://www.sota2.com/research/sota/question-answering-on-locomo-1-0-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-25T14:42:45.440Z

Evaluation of Question Answering performance on the LoCoMo 1.0 test set, focusing on F1 scores broken down by single-hop (S), multi-hop (M), open (O), and temporal (T) question types.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Question Answering |
| Dataset (including any stated split) | LoCoMo 1.0 (test) |
| Metric | F1 (Multi-Hop) |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 9 of 9 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [F1 (Multi-Hop)](<https://www.sota2.com/research/sota/question-answering-on-locomo-1-0-test.md?metric=F1%20(Multi-Hop)>): Higher is better; unit: PERCENT.

- [F1 (Single-Hop)](<https://www.sota2.com/research/sota/question-answering-on-locomo-1-0-test.md?metric=F1%20(Single-Hop)>): Higher is better; unit: PERCENT.

- [F1 (Open)](<https://www.sota2.com/research/sota/question-answering-on-locomo-1-0-test.md?metric=F1%20(Open)>): Higher is better; unit: PERCENT.

- [F1 (Temporal)](<https://www.sota2.com/research/sota/question-answering-on-locomo-1-0-test.md?metric=F1%20(Temporal)>): Higher is better; unit: PERCENT.

- [F1 (Overall)](<https://www.sota2.com/research/sota/question-answering-on-locomo-1-0-test.md?metric=F1%20(Overall)>): Higher is better; unit: PERCENT.

## Recorded results

| Method | F1 (Multi-Hop) | Result date | Report paper |
| --- | --- | --- | --- |
| Full-ctx 235B | 47 | Not recorded | 69c36c440636e5bf4afc66d1 |
| Mem0 (GPT-4o) | 38.7 | Not recorded | 69c36c440636e5bf4afc66d1 |
| Zep | 35.7 | Not recorded | 69c36c440636e5bf4afc66d1 |
| LangMem | 35.5 | Not recorded | 69c36c440636e5bf4afc66d1 |
| Warm (compound) | 20.8 | Not recorded | 69c36c440636e5bf4afc66d1 |
| Warm (memory only) | 20.8 | Not recorded | 69c36c440636e5bf4afc66d1 |
| Cold 8B | 8.8 | Not recorded | 69c36c440636e5bf4afc66d1 |
| Cold 235B | 7.6 | Not recorded | 69c36c440636e5bf4afc66d1 |
| Cold compound | 6.2 | Not recorded | 69c36c440636e5bf4afc66d1 |

### Full-ctx 235B

F1 (Multi-Hop): 47. Result ID: 69c3f45647f34ce924ade12e.

[Reporting paper](<https://www.sota2.com/research/paper/69c36c440636e5bf4afc66d1.md>)

Recorded evaluation configuration: {"Condition":"Full-ctx 235B","Mechanism":"All 19 sessions as conversation history"}. This record does not assert independent reproduction.

### Mem0 (GPT-4o)

F1 (Multi-Hop): 38.7. Result ID: 69c3f45647f34ce924ade12f.

[Reporting paper](<https://www.sota2.com/research/paper/69c36c440636e5bf4afc66d1.md>)

Recorded evaluation configuration: {"Condition":"Published baseline","Backbone":"GPT-4o"}. This record does not assert independent reproduction.

### Zep

F1 (Multi-Hop): 35.7. Result ID: 69c3f45647f34ce924ade130.

[Reporting paper](<https://www.sota2.com/research/paper/69c36c440636e5bf4afc66d1.md>)

Recorded evaluation configuration: {"Condition":"Published baseline"}. This record does not assert independent reproduction.

### LangMem

F1 (Multi-Hop): 35.5. Result ID: 69c3f45647f34ce924ade131.

[Reporting paper](<https://www.sota2.com/research/paper/69c36c440636e5bf4afc66d1.md>)

Recorded evaluation configuration: {"Condition":"Published baseline"}. This record does not assert independent reproduction.

### Warm (compound)

F1 (Multi-Hop): 20.8. Result ID: 69c3f45647f34ce924ade129.

[Reporting paper](<https://www.sota2.com/research/paper/69c36c440636e5bf4afc66d1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69c36c440636e5bf4afc66d1.md>)

Recorded evaluation configuration: {"Condition":"Warm (compound)","Mechanism":"Memory retrieval + confidence routing"}. This record does not assert independent reproduction.

### Warm (memory only)

F1 (Multi-Hop): 20.8. Result ID: 69c3f45647f34ce924ade12a.

[Reporting paper](<https://www.sota2.com/research/paper/69c36c440636e5bf4afc66d1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69c36c440636e5bf4afc66d1.md>)

Recorded evaluation configuration: {"Condition":"Warm (memory only)","Mechanism":"Memory retrieval to 8B, no routing"}. This record does not assert independent reproduction.

### Cold 8B

F1 (Multi-Hop): 8.8. Result ID: 69c3f45647f34ce924ade12c.

[Reporting paper](<https://www.sota2.com/research/paper/69c36c440636e5bf4afc66d1.md>)

Recorded evaluation configuration: {"Condition":"Cold 8B","Mechanism":"Direct 8B, no memory, no routing"}. This record does not assert independent reproduction.

### Cold 235B

F1 (Multi-Hop): 7.6. Result ID: 69c3f45647f34ce924ade12d.

[Reporting paper](<https://www.sota2.com/research/paper/69c36c440636e5bf4afc66d1.md>)

Recorded evaluation configuration: {"Condition":"Cold 235B","Mechanism":"Direct 235B, no memory, no routing"}. This record does not assert independent reproduction.

### Cold compound

F1 (Multi-Hop): 6.2. Result ID: 69c3f45647f34ce924ade12b.

[Reporting paper](<https://www.sota2.com/research/paper/69c36c440636e5bf4afc66d1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69c36c440636e5bf4afc66d1.md>)

Recorded evaluation configuration: {"Condition":"Cold compound","Mechanism":"No memory + confidence routing"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.