# Continual Language Modeling on HOPE 8-domain, high-overlap non-adaptive stream 256M

Source: [Continual Language Modeling on HOPE 8-domain, high-overlap non-adaptive stream 256M](<https://www.sota2.com/research/sota/continual-language-modeling-on-hope-8-domain-high-overlap-non-adaptive-stream-256m>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-27T07:19:46.048Z

Evaluation of Continual Language Modeling performance on the HOPE 8-domain dataset using the high-overlap non-adaptive stream configuration (256M tokens). Metrics focus on forgetting, average perplexity, and adaptation relative to a vanilla model.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Continual Language Modeling |
| Dataset (including any stated split) | HOPE 8-domain, high-overlap non-adaptive stream 256M |
| Metric | Forgetting |
| Unit | PERCENT |
| Direction | Lower is better |
| Results loaded | 6 of 6 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Forgetting](<https://www.sota2.com/research/sota/continual-language-modeling-on-hope-8-domain-high-overlap-non-adaptive-stream-256m.md?metric=Forgetting>): Lower is better; unit: PERCENT.

- [Avg PPL](<https://www.sota2.com/research/sota/continual-language-modeling-on-hope-8-domain-high-overlap-non-adaptive-stream-256m.md?metric=Avg%20PPL>): Lower is better; unit: SCALAR.

- [Forgetting Delta (vs Vanilla)](<https://www.sota2.com/research/sota/continual-language-modeling-on-hope-8-domain-high-overlap-non-adaptive-stream-256m.md?metric=Forgetting%20Delta%20(vs%20Vanilla)>): Lower is better; unit: SCALAR.

## Recorded results

| Method | Forgetting | Result date | Report paper |
| --- | --- | --- | --- |
| Adaptive-OGP | 9.4 | Not recorded | 69eeed5b55a3c32af89f959f |
| Replay 0.5% | 11.6 | Not recorded | 69eeed5b55a3c32af89f959f |
| Adam-NSCL | 12.5 | Not recorded | 69eeed5b55a3c32af89f959f |
| FOPNG | 12.8 | Not recorded | 69eeed5b55a3c32af89f959f |
| Vanilla | 13.2 | Not recorded | 69eeed5b55a3c32af89f959f |
| OGP (fixed α, no adapt) | 14.1 | Not recorded | 69eeed5b55a3c32af89f959f |

### Adaptive-OGP

Forgetting: 9.4. Result ID: 69ef0dea47f34ce924d4d233.

[Reporting paper](<https://www.sota2.com/research/paper/69eeed5b55a3c32af89f959f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69eeed5b55a3c32af89f959f.md>)

Recorded evaluation configuration: {"Routing":"decoupled"}. This record does not assert independent reproduction.

### Replay 0.5%

Forgetting: 11.6. Result ID: 69ef0dea47f34ce924d4d231.

[Reporting paper](<https://www.sota2.com/research/paper/69eeed5b55a3c32af89f959f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa63d71e209131cea01262.md>)

Recorded evaluation configuration: {"Routing":"shared"}. This record does not assert independent reproduction.

### Adam-NSCL

Forgetting: 12.5. Result ID: 69ef0dea47f34ce924d4d230.

[Reporting paper](<https://www.sota2.com/research/paper/69eeed5b55a3c32af89f959f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69931e209131cea22c9b.md>)

[Implementation](<https://github.com/ShipengWang/Adam-NSCL>)

Recorded evaluation configuration: {"Routing":"shared"}. This record does not assert independent reproduction.

### FOPNG

Forgetting: 12.8. Result ID: 69ef0dea47f34ce924d4d22f.

[Reporting paper](<https://www.sota2.com/research/paper/69eeed5b55a3c32af89f959f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69705d94e25ad2fa9d1ed177.md>)

Recorded evaluation configuration: {"Routing":"shared"}. This record does not assert independent reproduction.

### Vanilla

Forgetting: 13.2. Result ID: 69ef0dea47f34ce924d4d22e.

[Reporting paper](<https://www.sota2.com/research/paper/69eeed5b55a3c32af89f959f.md>)

Recorded evaluation configuration: {"Routing":"raw/raw"}. This record does not assert independent reproduction.

### OGP (fixed α, no adapt)

Forgetting: 14.1. Result ID: 69ef0dea47f34ce924d4d232.

[Reporting paper](<https://www.sota2.com/research/paper/69eeed5b55a3c32af89f959f.md>)

Recorded evaluation configuration: {"Routing":"decoupled"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.