# Gender bias evaluation on SEAT

Source: [Gender bias evaluation on SEAT](<https://www.sota2.com/research/sota/gender-bias-evaluation-on-seat>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-18T09:13:44.989Z

Evaluation of gender-related implicit associations using the Stereotype and Association Test (SEAT) methodology, reporting scores across various sub-tests (SEAT 6, 7, 8, etc.).

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Gender bias evaluation |
| Dataset (including any stated split) | SEAT |
| Metric | SEAT 6 |
| Unit | SCALAR |
| Direction | Lower is better |
| Results loaded | 16 of 16 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [SEAT 6](<https://www.sota2.com/research/sota/gender-bias-evaluation-on-seat.md?metric=SEAT%206>): Lower is better; unit: SCALAR.

- [SEAT 6b](<https://www.sota2.com/research/sota/gender-bias-evaluation-on-seat.md?metric=SEAT%206b>): Lower is better; unit: SCALAR.

- [SEAT 7](<https://www.sota2.com/research/sota/gender-bias-evaluation-on-seat.md?metric=SEAT%207>): Lower is better; unit: SCALAR.

- [SEAT 7b](<https://www.sota2.com/research/sota/gender-bias-evaluation-on-seat.md?metric=SEAT%207b>): Lower is better; unit: SCALAR.

- [SEAT 8](<https://www.sota2.com/research/sota/gender-bias-evaluation-on-seat.md?metric=SEAT%208>): Lower is better; unit: SCALAR.

- [SEAT 8b](<https://www.sota2.com/research/sota/gender-bias-evaluation-on-seat.md?metric=SEAT%208b>): Lower is better; unit: SCALAR.

- [Average SEAT Score](<https://www.sota2.com/research/sota/gender-bias-evaluation-on-seat.md?metric=Average%20SEAT%20Score>): Lower is better; unit: SCALAR.

## Recorded results

| Method | SEAT 6 | Result date | Report paper |
| --- | --- | --- | --- |
| CCPA | 0.181 | Not recorded | 68fa70001e209131cea487ac |
| CCPA | 0.251 | Not recorded | 68fa70001e209131cea487ac |
| Sent-Debias | 0.336 | Not recorded | 68fa70001e209131cea487ac |
| Auto-Debias | 0.373 | Not recorded | 68fa70001e209131cea487ac |
| Dropout | 0.38 | Not recorded | 6a0a9d35dbc836b87d82b56f |
| CCPA | 0.409 | Not recorded | 68fa70001e209131cea487ac |
| DebiasRAG | 0.42 | Not recorded | 6a0a9d35dbc836b87d82b56f |
| original | 0.48 | Not recorded | 6a0a9d35dbc836b87d82b56f |
| CDA | 0.596 | Not recorded | 68fa70001e209131cea487ac |
| INLP | 0.619 | Not recorded | 68fa70001e209131cea487ac |
| MABEL | 0.664 | Not recorded | 68fa70001e209131cea487ac |
| FairFil | 0.683 | Not recorded | 68fa70001e209131cea487ac |
| ELEATRA | 0.82 | Not recorded | 68fa70001e209131cea487ac |
| Dropout | 0.912 | Not recorded | 68fa70001e209131cea487ac |
| BERT | 0.932 | Not recorded | 68fa70001e209131cea487ac |
| DistilBERT | 1.38 | Not recorded | 68fa70001e209131cea487ac |

### CCPA

SEAT 6: 0.181. Result ID: 6991069a3a6c314849051a2d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70001e209131cea487ac.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70001e209131cea487ac.md>)

Recorded evaluation configuration: {"Backbone":"BERT","prompt\_template":"(1, 1, 1)"}. This record does not assert independent reproduction.

### CCPA

SEAT 6: 0.251. Result ID: 6991069a3a6c314849051a31.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70001e209131cea487ac.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70001e209131cea487ac.md>)

Recorded evaluation configuration: {"Backbone":"ELEATRA","prompt\_template":"(1, 1, 1)"}. This record does not assert independent reproduction.

### Sent-Debias

SEAT 6: 0.336. Result ID: 6991069a3a6c314849051a28.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70001e209131cea487ac.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa67f61e209131cea19723.md>)

Recorded evaluation configuration: {"Backbone":"BERT"}. This record does not assert independent reproduction.

### Auto-Debias

SEAT 6: 0.373. Result ID: 6991069a3a6c314849051a2a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70001e209131cea487ac.md>)

Recorded evaluation configuration: {"Backbone":"BERT"}. This record does not assert independent reproduction.

### Dropout

SEAT 6: 0.38. Result ID: 6a0ad83847f34ce924f13c0c.

[Reporting paper](<https://www.sota2.com/research/paper/6a0a9d35dbc836b87d82b56f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69751e209131cea22385.md>)

Recorded evaluation configuration: {"Backbone":"LLaMa3-8b"}. This record does not assert independent reproduction.

### CCPA

SEAT 6: 0.409. Result ID: 6991069a3a6c314849051a2f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70001e209131cea487ac.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70001e209131cea487ac.md>)

Recorded evaluation configuration: {"Backbone":"DistilBERT","prompt\_template":"(1, 1, 1)"}. This record does not assert independent reproduction.

### DebiasRAG

SEAT 6: 0.42. Result ID: 6a0ad83847f34ce924f13c0d.

[Reporting paper](<https://www.sota2.com/research/paper/6a0a9d35dbc836b87d82b56f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a0a9d35dbc836b87d82b56f.md>)

Recorded evaluation configuration: {"Backbone":"LLaMa3-8b"}. This record does not assert independent reproduction.

### original

SEAT 6: 0.48. Result ID: 6a0ad83847f34ce924f13c0b.

[Reporting paper](<https://www.sota2.com/research/paper/6a0a9d35dbc836b87d82b56f.md>)

Recorded evaluation configuration: {"Backbone":"LLaMa3-8b"}. This record does not assert independent reproduction.

### CDA

SEAT 6: 0.596. Result ID: 6991069a3a6c314849051a26.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70001e209131cea487ac.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa67941e209131cea17246.md>)

Recorded evaluation configuration: {"Backbone":"BERT"}. This record does not assert independent reproduction.

### INLP

SEAT 6: 0.619. Result ID: 6991069a3a6c314849051a2b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70001e209131cea487ac.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa676e1e209131cea16267.md>)

Recorded evaluation configuration: {"Backbone":"BERT"}. This record does not assert independent reproduction.

### MABEL

SEAT 6: 0.664. Result ID: 6991069a3a6c314849051a2c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70001e209131cea487ac.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6dc51e209131cea3c093.md>)

Recorded evaluation configuration: {"Backbone":"BERT"}. This record does not assert independent reproduction.

### FairFil

SEAT 6: 0.683. Result ID: 6991069a3a6c314849051a29.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70001e209131cea487ac.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa697f1e209131cea227b0.md>)

Recorded evaluation configuration: {"Backbone":"BERT"}. This record does not assert independent reproduction.

### ELEATRA

SEAT 6: 0.82. Result ID: 6991069a3a6c314849051a30.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70001e209131cea487ac.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa673f1e209131cea15373.md>)

[Implementation](<https://github.com/google-research/electra>)

Recorded evaluation configuration: {"Backbone":"ELEATRA"}. This record does not assert independent reproduction.

### Dropout

SEAT 6: 0.912. Result ID: 6991069a3a6c314849051a27.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70001e209131cea487ac.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69751e209131cea22385.md>)

Recorded evaluation configuration: {"Backbone":"BERT"}. This record does not assert independent reproduction.

### BERT

SEAT 6: 0.932. Result ID: 6991069a3a6c314849051a25.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70001e209131cea487ac.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa65c31e209131cea0c4ed.md>)

[Implementation](<https://github.com/google-research/bert>)

Recorded evaluation configuration: {"Backbone":"BERT"}. This record does not assert independent reproduction.

### DistilBERT

SEAT 6: 1.38. Result ID: 6991069a3a6c314849051a2e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70001e209131cea487ac.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa67221e209131cea14884.md>)

[Implementation](<https://github.com/huggingface/transformers>)

Recorded evaluation configuration: {"Backbone":"DistilBERT"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
