# agr\_refl\_num\_pp on CausalGym

Source: [agr\_refl\_num\_pp on CausalGym](<https://www.sota2.com/research/sota/agr_refl_num_pp-on-causalgym>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-11T09:59:40.775Z

Evaluation on the CausalGym dataset focusing on the 'agr\_refl\_num\_pp' task, measured using Completeness, Selectivity, and Reliability.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | agr\_refl\_num\_pp |
| Dataset (including any stated split) | CausalGym |
| Metric | Completeness |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 8 of 8 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Completeness](<https://www.sota2.com/research/sota/agr_refl_num_pp-on-causalgym.md?metric=Completeness>): Higher is better; unit: PERCENT.

- [Selectivity](<https://www.sota2.com/research/sota/agr_refl_num_pp-on-causalgym.md?metric=Selectivity>): Higher is better; unit: PERCENT.

- [Reliability](<https://www.sota2.com/research/sota/agr_refl_num_pp-on-causalgym.md?metric=Reliability>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Completeness | Result date | Report paper |
| --- | --- | --- | --- |
| HDMI | 100 | Not recorded | 6a0162b5dbc836b87d8232c6 |
| HDMI | 99.02 | Not recorded | 6a0162b5dbc836b87d8232c6 |
| AlterRep | 99 | Not recorded | 6a0162b5dbc836b87d8232c6 |
| FGSM | 96.42 | Not recorded | 6a0162b5dbc836b87d8232c6 |
| PGD | 61.42 | Not recorded | 6a0162b5dbc836b87d8232c6 |
| AlterRep | 54.49 | Not recorded | 6a0162b5dbc836b87d8232c6 |
| FGSM | 53.41 | Not recorded | 6a0162b5dbc836b87d8232c6 |
| PGD | 33.35 | Not recorded | 6a0162b5dbc836b87d8232c6 |

### HDMI

Completeness: 100. Result ID: 6a01a86547f34ce924e4067a.

[Reporting paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

Recorded evaluation configuration: {"Model":"EleutherAI/pythia-70m"}. This record does not assert independent reproduction.

### HDMI

Completeness: 99.02. Result ID: 6a01a86547f34ce924e40679.

[Reporting paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

Recorded evaluation configuration: {"Model":"Meta-Llama-3-8B-Instruct"}. This record does not assert independent reproduction.

### AlterRep

Completeness: 99. Result ID: 6a01a86547f34ce924e4067b.

[Reporting paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ad11e209131cea2a835.md>)

Recorded evaluation configuration: {"Model":"Meta-Llama-3-8B-Instruct"}. This record does not assert independent reproduction.

### FGSM

Completeness: 96.42. Result ID: 6a01a86547f34ce924e4067e.

[Reporting paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa61c91e209131ce9f5e06.md>)

Recorded evaluation configuration: {"Model":"EleutherAI/pythia-70m"}. This record does not assert independent reproduction.

### PGD

Completeness: 61.42. Result ID: 6a01a86547f34ce924e40680.

[Reporting paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa665a1e209131cea0fbae.md>)

Recorded evaluation configuration: {"Model":"EleutherAI/pythia-70m"}. This record does not assert independent reproduction.

### AlterRep

Completeness: 54.49. Result ID: 6a01a86547f34ce924e4067c.

[Reporting paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ad11e209131cea2a835.md>)

Recorded evaluation configuration: {"Model":"EleutherAI/pythia-70m"}. This record does not assert independent reproduction.

### FGSM

Completeness: 53.41. Result ID: 6a01a86547f34ce924e4067d.

[Reporting paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa61c91e209131ce9f5e06.md>)

Recorded evaluation configuration: {"Model":"Meta-Llama-3-8B-Instruct"}. This record does not assert independent reproduction.

### PGD

Completeness: 33.35. Result ID: 6a01a86547f34ce924e4067f.

[Reporting paper](<https://www.sota2.com/research/paper/6a0162b5dbc836b87d8232c6.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa665a1e209131cea0fbae.md>)

Recorded evaluation configuration: {"Model":"Meta-Llama-3-8B-Instruct"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.