# Filler Gap Subject Dependency on CausalGym

Source: [Filler Gap Subject Dependency on CausalGym](<https://www.sota2.com/research/sota/filler-gap-subject-dependency-on-causalgym>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-11T09:59:40.902Z

Evaluation of model performance on the filler\_gap\_subj task using the CausalGym dataset, measured by Completeness, Selectivity, and Reliability.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | filler\_gap\_subj |
| Dataset (including any stated split) | CausalGym |
| Metric | Completeness |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 8 of 8 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Completeness](<https://www.sota2.com/research/sota/filler-gap-subject-dependency-on-causalgym.md?metric=Completeness>): Higher is better; unit: PERCENT.

- [Selectivity](<https://www.sota2.com/research/sota/filler-gap-subject-dependency-on-causalgym.md?metric=Selectivity>): Higher is better; unit: PERCENT.

- [Reliability](<https://www.sota2.com/research/sota/filler-gap-subject-dependency-on-causalgym.md?metric=Reliability>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Completeness | Result date | Report paper |
| --- | --- | --- | --- |
| HDMI | 100 | Not recorded | 6a0162b5dbc836b87d8232c6 |
| HDMI | 100 | Not recorded | 6a0162b5dbc836b87d8232c6 |
| AlterRep | 100 | Not recorded | 6a0162b5dbc836b87d8232c6 |
| AlterRep | 100 | Not recorded | 6a0162b5dbc836b87d8232c6 |
| FGSM | 92 | Not recorded | 6a0162b5dbc836b87d8232c6 |
| PGD | 90.5 | Not recorded | 6a0162b5dbc836b87d8232c6 |
| FGSM | 34.35 | Not recorded | 6a0162b5dbc836b87d8232c6 |
| PGD | 17.87 | Not recorded | 6a0162b5dbc836b87d8232c6 |

### HDMI

Completeness: 100. Result ID: 6a01a86547f34ce924e406a9.

[Reporting paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

Recorded evaluation configuration: {"Model":"Meta-Llama-3-8B-Instruct"}. This record does not assert independent reproduction.

### HDMI

Completeness: 100. Result ID: 6a01a86547f34ce924e406aa.

[Reporting paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

Recorded evaluation configuration: {"Model":"EleutherAI/pythia-70m"}. This record does not assert independent reproduction.

### AlterRep

Completeness: 100. Result ID: 6a01a86547f34ce924e406ab.

[Reporting paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ad11e209131cea2a835.md>)

Recorded evaluation configuration: {"Model":"Meta-Llama-3-8B-Instruct"}. This record does not assert independent reproduction.

### AlterRep

Completeness: 100. Result ID: 6a01a86547f34ce924e406ac.

[Reporting paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ad11e209131cea2a835.md>)

Recorded evaluation configuration: {"Model":"EleutherAI/pythia-70m"}. This record does not assert independent reproduction.

### FGSM

Completeness: 92. Result ID: 6a01a86547f34ce924e406ad.

[Reporting paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa61c91e209131ce9f5e06.md>)

Recorded evaluation configuration: {"Model":"Meta-Llama-3-8B-Instruct"}. This record does not assert independent reproduction.

### PGD

Completeness: 90.5. Result ID: 6a01a86547f34ce924e406af.

[Reporting paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa665a1e209131cea0fbae.md>)

Recorded evaluation configuration: {"Model":"Meta-Llama-3-8B-Instruct"}. This record does not assert independent reproduction.

### FGSM

Completeness: 34.35. Result ID: 6a01a86547f34ce924e406ae.

[Reporting paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa61c91e209131ce9f5e06.md>)

Recorded evaluation configuration: {"Model":"EleutherAI/pythia-70m"}. This record does not assert independent reproduction.

### PGD

Completeness: 17.87. Result ID: 6a01a86547f34ce924e406b0.

[Reporting paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa665a1e209131cea0fbae.md>)

Recorded evaluation configuration: {"Model":"EleutherAI/pythia-70m"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.