# Syntactic Surprisal Update Measurement (KL-divergence) on SAP All (full) Source: [Syntactic Surprisal Update Measurement (KL-divergence) on SAP All (full)]() Collection index: [llms.txt]() Data updated: 2026-04-21T13:42:08.903Z Evaluation of model performance on measuring Syntactic Surprisal Updates using KL-divergence on the full SAP All dataset, utilizing metrics like MVRR, NPS, and NPZ. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Syntactic Surprisal Update Measurement (KL-divergence) | | Dataset (including any stated split) | SAP All (full) | | Metric | MVRR | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 19 of 19 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [MVRR](): Higher is better; unit: SCALAR. - [NPS](): Higher is better; unit: SCALAR. - [NPZ](): Higher is better; unit: SCALAR. - [RC](): Higher is better; unit: SCALAR. - [Att](): Higher is better; unit: SCALAR. ## Recorded results | Method | MVRR | Result date | Report paper | | --- | --- | --- | --- | | PT-1b | 38.5 | Not recorded | 69e705337d0172140ec7f3b0 | | PT-160m | 24.5 | Not recorded | 69e705337d0172140ec7f3b0 | | PT-2.8b | 20.3 | Not recorded | 69e705337d0172140ec7f3b0 | | PT-70m | 10.6 | Not recorded | 69e705337d0172140ec7f3b0 | | G2-sm | 10 | Not recorded | 69e705337d0172140ec7f3b0 | | PT-410m | 9.9 | Not recorded | 69e705337d0172140ec7f3b0 | | G2-md | 9.7 | Not recorded | 69e705337d0172140ec7f3b0 | | OT-1.3b | 9.6 | Not recorded | 69e705337d0172140ec7f3b0 | | OT-2.7b | 9.5 | Not recorded | 69e705337d0172140ec7f3b0 | | OT-125m | 7.7 | Not recorded | 69e705337d0172140ec7f3b0 | | PT-1.4b | 7.6 | Not recorded | 69e705337d0172140ec7f3b0 | | G2-lg | 7.4 | Not recorded | 69e705337d0172140ec7f3b0 | | OT-13b | 6.5 | Not recorded | 69e705337d0172140ec7f3b0 | | OT-30b | 4.8 | Not recorded | 69e705337d0172140ec7f3b0 | | PT-12b | 4.5 | Not recorded | 69e705337d0172140ec7f3b0 | | PT-6.9b | 4.3 | Not recorded | 69e705337d0172140ec7f3b0 | | G2-xl | 3.1 | Not recorded | 69e705337d0172140ec7f3b0 | | OT-6.7b | 2.7 | Not recorded | 69e705337d0172140ec7f3b0 | | OT-66b | 2.2 | Not recorded | 69e705337d0172140ec7f3b0 | ### PT-1b MVRR: 38.5. Result ID: 69e77e9447f34ce924cf81cb. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"Pythia"}. This record does not assert independent reproduction. ### PT-160m MVRR: 24.5. Result ID: 69e77e9447f34ce924cf81c9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"Pythia"}. This record does not assert independent reproduction. ### PT-2.8b MVRR: 20.3. Result ID: 69e77e9447f34ce924cf81cd. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"Pythia"}. This record does not assert independent reproduction. ### PT-70m MVRR: 10.6. Result ID: 69e77e9447f34ce924cf81c8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"Pythia"}. This record does not assert independent reproduction. ### G2-sm MVRR: 10. Result ID: 69e77e9447f34ce924cf81bd. [Reporting paper]() Recorded evaluation configuration: {"Family":"GPT-2"}. This record does not assert independent reproduction. ### PT-410m MVRR: 9.9. Result ID: 69e77e9447f34ce924cf81ca. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"Pythia"}. This record does not assert independent reproduction. ### G2-md MVRR: 9.7. Result ID: 69e77e9447f34ce924cf81be. [Reporting paper]() Recorded evaluation configuration: {"Family":"GPT-2"}. This record does not assert independent reproduction. ### OT-1.3b MVRR: 9.6. Result ID: 69e77e9447f34ce924cf81c2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"OPT"}. This record does not assert independent reproduction. ### OT-2.7b MVRR: 9.5. Result ID: 69e77e9447f34ce924cf81c3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"OPT"}. This record does not assert independent reproduction. ### OT-125m MVRR: 7.7. Result ID: 69e77e9447f34ce924cf81c1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"OPT"}. This record does not assert independent reproduction. ### PT-1.4b MVRR: 7.6. Result ID: 69e77e9447f34ce924cf81cc. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"Pythia"}. This record does not assert independent reproduction. ### G2-lg MVRR: 7.4. Result ID: 69e77e9447f34ce924cf81bf. [Reporting paper]() Recorded evaluation configuration: {"Family":"GPT-2"}. This record does not assert independent reproduction. ### OT-13b MVRR: 6.5. Result ID: 69e77e9447f34ce924cf81c5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"OPT"}. This record does not assert independent reproduction. ### OT-30b MVRR: 4.8. Result ID: 69e77e9447f34ce924cf81c6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"OPT"}. This record does not assert independent reproduction. ### PT-12b MVRR: 4.5. Result ID: 69e77e9447f34ce924cf81cf. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"Pythia"}. This record does not assert independent reproduction. ### PT-6.9b MVRR: 4.3. Result ID: 69e77e9447f34ce924cf81ce. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"Pythia"}. This record does not assert independent reproduction. ### G2-xl MVRR: 3.1. Result ID: 69e77e9447f34ce924cf81c0. [Reporting paper]() Recorded evaluation configuration: {"Family":"GPT-2"}. This record does not assert independent reproduction. ### OT-6.7b MVRR: 2.7. Result ID: 69e77e9447f34ce924cf81c4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"OPT"}. This record does not assert independent reproduction. ### OT-66b MVRR: 2.2. Result ID: 69e77e9447f34ce924cf81c7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"OPT"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.