# Long-term Memory and Reasoning on Grocery Shopping CSQA Source (train and dev)

Source: [Long-term Memory and Reasoning on Grocery Shopping CSQA Source (train and dev)](<https://www.sota2.com/research/sota/long-term-memory-and-reasoning-on-grocery-shopping-csqa-source-train-and-dev>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:21:38.899Z

Evaluation of model performance on the Grocery Shopping CSQA source dataset (train and dev splits), focusing on long-term memory and reasoning capabilities, measured by ACCc, ACCg, Δc, and Δg.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Long-term memory and reasoning |
| Dataset (including any stated split) | Grocery Shopping CSQA source (train and dev) |
| Metric | ACCc |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 12 of 12 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [ACCc](<https://www.sota2.com/research/sota/long-term-memory-and-reasoning-on-grocery-shopping-csqa-source-train-and-dev.md?metric=ACCc>): Higher is better; unit: PERCENT.

- [ACCg](<https://www.sota2.com/research/sota/long-term-memory-and-reasoning-on-grocery-shopping-csqa-source-train-and-dev.md?metric=ACCg>): Higher is better; unit: PERCENT.

- [Delta c (Δc)](<https://www.sota2.com/research/sota/long-term-memory-and-reasoning-on-grocery-shopping-csqa-source-train-and-dev.md?metric=Delta%20c%20(%CE%94c)>): Higher is better; unit: SCALAR.

- [Delta g (Δg)](<https://www.sota2.com/research/sota/long-term-memory-and-reasoning-on-grocery-shopping-csqa-source-train-and-dev.md?metric=Delta%20g%20(%CE%94g)>): Higher is better; unit: SCALAR.

## Recorded results

| Method | ACCc | Result date | Report paper |
| --- | --- | --- | --- |
| SirLLM | 81.44 | Not recorded | 68fa733f1e209131cea5b3c3 |
| StreamLLM | 81.35 | Not recorded | 68fa733f1e209131cea5b3c3 |
| SirLLM | 72.44 | Not recorded | 68fa733f1e209131cea5b3c3 |
| StreamLLM | 71.33 | Not recorded | 68fa733f1e209131cea5b3c3 |
| RandomLLM | 70.33 | Not recorded | 68fa733f1e209131cea5b3c3 |
| IntervalLLM | 63.98 | Not recorded | 68fa733f1e209131cea5b3c3 |
| SirLLM | 60.23 | Not recorded | 68fa733f1e209131cea5b3c3 |
| StreamLLM | 60.1 | Not recorded | 68fa733f1e209131cea5b3c3 |
| SirLLM | 51.04 | Not recorded | 68fa733f1e209131cea5b3c3 |
| RandomLLM | 50.97 | Not recorded | 68fa733f1e209131cea5b3c3 |
| StreamLLM | 50.84 | Not recorded | 68fa733f1e209131cea5b3c3 |
| IntervalLLM | 47.21 | Not recorded | 68fa733f1e209131cea5b3c3 |

### SirLLM

ACCc: 81.44. Result ID: 6992cc603a6c314849119239.

[Reporting paper](<https://www.sota2.com/research/paper/68fa733f1e209131cea5b3c3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa733f1e209131cea5b3c3.md>)

[Implementation](<https://github.com/Zoeyyao27/SirLLM>)

Recorded evaluation configuration: {"LLM Backbone":"Yi-34b","# Entropy":1020,"# Recent":0,"Attention Sink Size":4,"Ndecay":1}. This record does not assert independent reproduction.

### StreamLLM

ACCc: 81.35. Result ID: 6992cc603a6c314849119238.

[Reporting paper](<https://www.sota2.com/research/paper/68fa733f1e209131cea5b3c3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72d71e209131cea58d79.md>)

Recorded evaluation configuration: {"LLM Backbone":"Yi-34b","# Entropy":0,"# Recent":1020,"Attention Sink Size":4,"Ndecay":1}. This record does not assert independent reproduction.

### SirLLM

ACCc: 72.44. Result ID: 6992cc603a6c314849119231.

[Reporting paper](<https://www.sota2.com/research/paper/68fa733f1e209131cea5b3c3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa733f1e209131cea5b3c3.md>)

[Implementation](<https://github.com/Zoeyyao27/SirLLM>)

Recorded evaluation configuration: {"LLM Backbone":"Yi-6b","# Entropy":1020,"# Recent":0,"Attention Sink Size":4,"Ndecay":1}. This record does not assert independent reproduction.

### StreamLLM

ACCc: 71.33. Result ID: 6992cc603a6c31484911922e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa733f1e209131cea5b3c3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72d71e209131cea58d79.md>)

Recorded evaluation configuration: {"LLM Backbone":"Yi-6b","# Entropy":0,"# Recent":1020,"Attention Sink Size":4,"Ndecay":1}. This record does not assert independent reproduction.

### RandomLLM

ACCc: 70.33. Result ID: 6992cc603a6c31484911922f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa733f1e209131cea5b3c3.md>)

Recorded evaluation configuration: {"LLM Backbone":"Yi-6b","# Entropy":1020,"# Recent":0,"Attention Sink Size":4,"Ndecay":1}. This record does not assert independent reproduction.

### IntervalLLM

ACCc: 63.98. Result ID: 6992cc603a6c314849119230.

[Reporting paper](<https://www.sota2.com/research/paper/68fa733f1e209131cea5b3c3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa65aa1e209131cea0bc26.md>)

Recorded evaluation configuration: {"LLM Backbone":"Yi-6b","# Entropy":1020,"# Recent":0,"Attention Sink Size":4,"Ndecay":1}. This record does not assert independent reproduction.

### SirLLM

ACCc: 60.23. Result ID: 6992cc603a6c314849119237.

[Reporting paper](<https://www.sota2.com/research/paper/68fa733f1e209131cea5b3c3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa733f1e209131cea5b3c3.md>)

[Implementation](<https://github.com/Zoeyyao27/SirLLM>)

Recorded evaluation configuration: {"LLM Backbone":"Vicuna-13b","# Entropy":1020,"# Recent":0,"Attention Sink Size":4,"Ndecay":1}. This record does not assert independent reproduction.

### StreamLLM

ACCc: 60.1. Result ID: 6992cc603a6c314849119236.

[Reporting paper](<https://www.sota2.com/research/paper/68fa733f1e209131cea5b3c3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72d71e209131cea58d79.md>)

Recorded evaluation configuration: {"LLM Backbone":"Vicuna-13b","# Entropy":0,"# Recent":1020,"Attention Sink Size":4,"Ndecay":1}. This record does not assert independent reproduction.

### SirLLM

ACCc: 51.04. Result ID: 6992cc603a6c314849119235.

[Reporting paper](<https://www.sota2.com/research/paper/68fa733f1e209131cea5b3c3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa733f1e209131cea5b3c3.md>)

[Implementation](<https://github.com/Zoeyyao27/SirLLM>)

Recorded evaluation configuration: {"LLM Backbone":"Vicuna-7b","# Entropy":1020,"# Recent":0,"Attention Sink Size":4,"Ndecay":1}. This record does not assert independent reproduction.

### RandomLLM

ACCc: 50.97. Result ID: 6992cc603a6c314849119233.

[Reporting paper](<https://www.sota2.com/research/paper/68fa733f1e209131cea5b3c3.md>)

Recorded evaluation configuration: {"LLM Backbone":"Vicuna-7b","# Entropy":1020,"# Recent":0,"Attention Sink Size":4,"Ndecay":1}. This record does not assert independent reproduction.

### StreamLLM

ACCc: 50.84. Result ID: 6992cc603a6c314849119232.

[Reporting paper](<https://www.sota2.com/research/paper/68fa733f1e209131cea5b3c3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72d71e209131cea58d79.md>)

Recorded evaluation configuration: {"LLM Backbone":"Vicuna-7b","# Entropy":0,"# Recent":1020,"Attention Sink Size":4,"Ndecay":1}. This record does not assert independent reproduction.

### IntervalLLM

ACCc: 47.21. Result ID: 6992cc603a6c314849119234.

[Reporting paper](<https://www.sota2.com/research/paper/68fa733f1e209131cea5b3c3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa65aa1e209131cea0bc26.md>)

Recorded evaluation configuration: {"LLM Backbone":"Vicuna-7b","# Entropy":1020,"# Recent":0,"Attention Sink Size":4,"Ndecay":1}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
