# Psychometric Predictive Power on SAP ROI Source: [Psychometric Predictive Power on SAP ROI]() Collection index: [llms.txt]() Data updated: 2026-04-21T13:42:08.874Z Evaluation of model performance on psychometric predictive power using the SAP ROI dataset, assessed using metrics such as MVRR, NPS, NPZ, RC, and Att. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Psychometric Predictive Power | | Dataset (including any stated split) | SAP ROI | | Metric | MVRR | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 38 of 38 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [MVRR](): Higher is better; unit: SCALAR. - [NPS](): Higher is better; unit: SCALAR. - [NPZ](): Higher is better; unit: SCALAR. - [RC](): Higher is better; unit: SCALAR. - [Att.](): Higher is better; unit: SCALAR. ## Recorded results | Method | MVRR | Result date | Report paper | | --- | --- | --- | --- | | surprisal update | 14.1 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 13.5 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 11.1 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 11 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 10.4 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 9.1 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 8.8 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 8.7 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 8.7 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 8.6 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 7.8 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 7.7 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 7.1 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 7 | Not recorded | 69e705337d0172140ec7f3b0 | | OT-30b | 5.9 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 5.5 | Not recorded | 69e705337d0172140ec7f3b0 | | OT-13b | 4.6 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 3.7 | Not recorded | 69e705337d0172140ec7f3b0 | | PT-1b | 3.2 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 3.1 | Not recorded | 69e705337d0172140ec7f3b0 | | OT-6.7b | 2.9 | Not recorded | 69e705337d0172140ec7f3b0 | | PT-410m | 2.9 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 2.6 | Not recorded | 69e705337d0172140ec7f3b0 | | G2-sm | 2.5 | Not recorded | 69e705337d0172140ec7f3b0 | | G2-lg | 2.2 | Not recorded | 69e705337d0172140ec7f3b0 | | G2-xl | 2.2 | Not recorded | 69e705337d0172140ec7f3b0 | | G2-md | 2 | Not recorded | 69e705337d0172140ec7f3b0 | | PT-70m | 2 | Not recorded | 69e705337d0172140ec7f3b0 | | OT-2.7b | 1.9 | Not recorded | 69e705337d0172140ec7f3b0 | | OT-66b | 1.7 | Not recorded | 69e705337d0172140ec7f3b0 | | surprisal update | 1.6 | Not recorded | 69e705337d0172140ec7f3b0 | | OT-1.3b | 1.6 | Not recorded | 69e705337d0172140ec7f3b0 | | PT-2.8b | 0.7 | Not recorded | 69e705337d0172140ec7f3b0 | | PT-160m | 0.5 | Not recorded | 69e705337d0172140ec7f3b0 | | PT-1.4b | 0.3 | Not recorded | 69e705337d0172140ec7f3b0 | | PT-12b | 0.3 | Not recorded | 69e705337d0172140ec7f3b0 | | OT-125m | 0.2 | Not recorded | 69e705337d0172140ec7f3b0 | | PT-6.9b | 0.2 | Not recorded | 69e705337d0172140ec7f3b0 | ### surprisal update MVRR: 14.1. Result ID: 69e77e9447f34ce924cf81aa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"G2-sm"}. This record does not assert independent reproduction. ### surprisal update MVRR: 13.5. Result ID: 69e77e9447f34ce924cf81ae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"OT-125m"}. This record does not assert independent reproduction. ### surprisal update MVRR: 11.1. Result ID: 69e77e9447f34ce924cf81b2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"OT-13b"}. This record does not assert independent reproduction. ### surprisal update MVRR: 11. Result ID: 69e77e9447f34ce924cf81b0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"OT-2.7b"}. This record does not assert independent reproduction. ### surprisal update MVRR: 10.4. Result ID: 69e77e9447f34ce924cf81bc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"PT-12b"}. This record does not assert independent reproduction. ### surprisal update MVRR: 9.1. Result ID: 69e77e9447f34ce924cf81b9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"PT-1.4b"}. This record does not assert independent reproduction. ### surprisal update MVRR: 8.8. Result ID: 69e77e9447f34ce924cf81b7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"PT-410m"}. This record does not assert independent reproduction. ### surprisal update MVRR: 8.7. Result ID: 69e77e9447f34ce924cf81ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"G2-md"}. This record does not assert independent reproduction. ### surprisal update MVRR: 8.7. Result ID: 69e77e9447f34ce924cf81b8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"PT-1b"}. This record does not assert independent reproduction. ### surprisal update MVRR: 8.6. Result ID: 69e77e9447f34ce924cf81bb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"PT-6.9b"}. This record does not assert independent reproduction. ### surprisal update MVRR: 7.8. Result ID: 69e77e9447f34ce924cf81af. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"OT-1.3b"}. This record does not assert independent reproduction. ### surprisal update MVRR: 7.7. Result ID: 69e77e9447f34ce924cf81b4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"OT-66b"}. This record does not assert independent reproduction. ### surprisal update MVRR: 7.1. Result ID: 69e77e9447f34ce924cf81b1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"OT-6.7b"}. This record does not assert independent reproduction. ### surprisal update MVRR: 7. Result ID: 69e77e9447f34ce924cf81ac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"G2-lg"}. This record does not assert independent reproduction. ### OT-30b MVRR: 5.9. Result ID: 69e77e9447f34ce924cf81d9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"OPT"}. This record does not assert independent reproduction. ### surprisal update MVRR: 5.5. Result ID: 69e77e9447f34ce924cf81ad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"G2-xl"}. This record does not assert independent reproduction. ### OT-13b MVRR: 4.6. Result ID: 69e77e9447f34ce924cf81d8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"OPT"}. This record does not assert independent reproduction. ### surprisal update MVRR: 3.7. Result ID: 69e77e9447f34ce924cf81ba. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"PT-2.8b"}. This record does not assert independent reproduction. ### PT-1b MVRR: 3.2. Result ID: 69e77e9447f34ce924cf81de. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"Pythia"}. This record does not assert independent reproduction. ### surprisal update MVRR: 3.1. Result ID: 69e77e9447f34ce924cf81b3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"OT-30b"}. This record does not assert independent reproduction. ### OT-6.7b MVRR: 2.9. Result ID: 69e77e9447f34ce924cf81d7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"OPT"}. This record does not assert independent reproduction. ### PT-410m MVRR: 2.9. Result ID: 69e77e9447f34ce924cf81dd. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"Pythia"}. This record does not assert independent reproduction. ### surprisal update MVRR: 2.6. Result ID: 69e77e9447f34ce924cf81b5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"PT-70m"}. This record does not assert independent reproduction. ### G2-sm MVRR: 2.5. Result ID: 69e77e9447f34ce924cf81d0. [Reporting paper]() Recorded evaluation configuration: {"Family":"GPT-2"}. This record does not assert independent reproduction. ### G2-lg MVRR: 2.2. Result ID: 69e77e9447f34ce924cf81d2. [Reporting paper]() Recorded evaluation configuration: {"Family":"GPT-2"}. This record does not assert independent reproduction. ### G2-xl MVRR: 2.2. Result ID: 69e77e9447f34ce924cf81d3. [Reporting paper]() Recorded evaluation configuration: {"Family":"GPT-2"}. This record does not assert independent reproduction. ### G2-md MVRR: 2. Result ID: 69e77e9447f34ce924cf81d1. [Reporting paper]() Recorded evaluation configuration: {"Family":"GPT-2"}. This record does not assert independent reproduction. ### PT-70m MVRR: 2. Result ID: 69e77e9447f34ce924cf81db. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"Pythia"}. This record does not assert independent reproduction. ### OT-2.7b MVRR: 1.9. Result ID: 69e77e9447f34ce924cf81d6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"OPT"}. This record does not assert independent reproduction. ### OT-66b MVRR: 1.7. Result ID: 69e77e9447f34ce924cf81da. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"OPT"}. This record does not assert independent reproduction. ### surprisal update MVRR: 1.6. Result ID: 69e77e9447f34ce924cf81b6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"PT-160m"}. This record does not assert independent reproduction. ### OT-1.3b MVRR: 1.6. Result ID: 69e77e9447f34ce924cf81d5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"OPT"}. This record does not assert independent reproduction. ### PT-2.8b MVRR: 0.7. Result ID: 69e77e9447f34ce924cf81e0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"Pythia"}. This record does not assert independent reproduction. ### PT-160m MVRR: 0.5. Result ID: 69e77e9447f34ce924cf81dc. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"Pythia"}. This record does not assert independent reproduction. ### PT-1.4b MVRR: 0.3. Result ID: 69e77e9447f34ce924cf81df. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"Pythia"}. This record does not assert independent reproduction. ### PT-12b MVRR: 0.3. Result ID: 69e77e9447f34ce924cf81e2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"Pythia"}. This record does not assert independent reproduction. ### OT-125m MVRR: 0.2. Result ID: 69e77e9447f34ce924cf81d4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"OPT"}. This record does not assert independent reproduction. ### PT-6.9b MVRR: 0.2. Result ID: 69e77e9447f34ce924cf81e1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Family":"Pythia"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.