# Subject-verb number agreement in object-relative clauses on CausalGym agr\_sv\_num\_obj-relc Source: [Subject-verb number agreement in object-relative clauses on CausalGym agr\_sv\_num\_obj-relc]() Collection index: [llms.txt]() Data updated: 2026-05-11T09:59:40.500Z Evaluation of model performance on subject-verb number agreement specifically within object-relative clauses, using the CausalGym benchmark subset, measured by Completeness, Selectivity, and Reliability. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Subject-verb number agreement in object-relative clauses | | Dataset (including any stated split) | CausalGym agr\_sv\_num\_obj-relc | | Metric | Completeness | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Completeness](): Higher is better; unit: PERCENT. - [Selectivity](): Higher is better; unit: PERCENT. - [Reliability](): Higher is better; unit: PERCENT. ## Recorded results | Method | Completeness | Result date | Report paper | | --- | --- | --- | --- | | HDMI | 100 | Not recorded | 6a0162b5dbc836b87d8232c6 | | FGSM | 100 | Not recorded | 6a0162b5dbc836b87d8232c6 | | HDMI | 100 | Not recorded | 6a0162b5dbc836b87d8232c6 | | FGSM | 100 | Not recorded | 6a0162b5dbc836b87d8232c6 | | PGD | 100 | Not recorded | 6a0162b5dbc836b87d8232c6 | | HDMI | 100 | Not recorded | 6a0162b5dbc836b87d8232c6 | | HDMI | 100 | Not recorded | 6a0162b5dbc836b87d8232c6 | | FGSM | 100 | Not recorded | 6a0162b5dbc836b87d8232c6 | | FGSM | 100 | Not recorded | 6a0162b5dbc836b87d8232c6 | | PGD | 100 | Not recorded | 6a0162b5dbc836b87d8232c6 | | AlterRep | 98 | Not recorded | 6a0162b5dbc836b87d8232c6 | | AlterRep | 98 | Not recorded | 6a0162b5dbc836b87d8232c6 | | AlterRep | 96 | Not recorded | 6a0162b5dbc836b87d8232c6 | | AlterRep | 96 | Not recorded | 6a0162b5dbc836b87d8232c6 | | PGD | 63.07 | Not recorded | 6a0162b5dbc836b87d8232c6 | | PGD | 63.07 | Not recorded | 6a0162b5dbc836b87d8232c6 | ### HDMI Completeness: 100. Result ID: 6a01a86547f34ce924e40631. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Meta-Llama-3-8B-Instruct"}. This record does not assert independent reproduction. ### FGSM Completeness: 100. Result ID: 6a01a86547f34ce924e40633. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Meta-Llama-3-8B-Instruct"}. This record does not assert independent reproduction. ### HDMI Completeness: 100. Result ID: 6a01a86547f34ce924e40635. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"EleutherAI/pythia-70m"}. This record does not assert independent reproduction. ### FGSM Completeness: 100. Result ID: 6a01a86547f34ce924e40637. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"EleutherAI/pythia-70m"}. This record does not assert independent reproduction. ### PGD Completeness: 100. Result ID: 6a01a86547f34ce924e40638. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"EleutherAI/pythia-70m"}. This record does not assert independent reproduction. ### HDMI Completeness: 100. Result ID: 6a01a86547f34ce924e40661. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Meta-Llama-3-8B-Instruct"}. This record does not assert independent reproduction. ### HDMI Completeness: 100. Result ID: 6a01a86547f34ce924e40662. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"EleutherAI/pythia-70m"}. This record does not assert independent reproduction. ### FGSM Completeness: 100. Result ID: 6a01a86547f34ce924e40665. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Meta-Llama-3-8B-Instruct"}. This record does not assert independent reproduction. ### FGSM Completeness: 100. Result ID: 6a01a86547f34ce924e40666. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"EleutherAI/pythia-70m"}. This record does not assert independent reproduction. ### PGD Completeness: 100. Result ID: 6a01a86547f34ce924e40668. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"EleutherAI/pythia-70m"}. This record does not assert independent reproduction. ### AlterRep Completeness: 98. Result ID: 6a01a86547f34ce924e40632. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Meta-Llama-3-8B-Instruct"}. This record does not assert independent reproduction. ### AlterRep Completeness: 98. Result ID: 6a01a86547f34ce924e40663. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Meta-Llama-3-8B-Instruct"}. This record does not assert independent reproduction. ### AlterRep Completeness: 96. Result ID: 6a01a86547f34ce924e40636. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"EleutherAI/pythia-70m"}. This record does not assert independent reproduction. ### AlterRep Completeness: 96. Result ID: 6a01a86547f34ce924e40664. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"EleutherAI/pythia-70m"}. This record does not assert independent reproduction. ### PGD Completeness: 63.07. Result ID: 6a01a86547f34ce924e40634. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Meta-Llama-3-8B-Instruct"}. This record does not assert independent reproduction. ### PGD Completeness: 63.07. Result ID: 6a01a86547f34ce924e40667. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Meta-Llama-3-8B-Instruct"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.