# Sentiment Classification on REAL-AAE (held-out evaluation set)

Source: [Sentiment Classification on REAL-AAE (held-out evaluation set)](<https://www.sota2.com/research/sota/sentiment-classification-on-real-aae-held-out-evaluation-set>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-07-09T06:57:49.788Z

Evaluation of Sentiment Classification performance on the REAL-AAE held-out evaluation set, measured using DGI, Two-way DGI, and cDGI metrics.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Sentiment Classification |
| Dataset (including any stated split) | REAL-AAE (held-out evaluation set) |
| Metric | DGI |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 6 of 6 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [DGI](<https://www.sota2.com/research/sota/sentiment-classification-on-real-aae-held-out-evaluation-set.md?metric=DGI>): Higher is better; unit: PERCENT.

- [Two-way DGI](<https://www.sota2.com/research/sota/sentiment-classification-on-real-aae-held-out-evaluation-set.md?metric=Two-way%20DGI>): Higher is better; unit: PERCENT.

- [cDGI](<https://www.sota2.com/research/sota/sentiment-classification-on-real-aae-held-out-evaluation-set.md?metric=cDGI>): Higher is better; unit: PERCENT.

## Recorded results

| Method | DGI | Result date | Report paper |
| --- | --- | --- | --- |
| Gemma-3 | 84.9 | Not recorded | 6a4f2b1a32fb937314875c0a |
| Phi-4 | 81.4 | Not recorded | 6a4f2b1a32fb937314875c0a |
| Mistral-Small-3.1-24B | 79.5 | Not recorded | 6a4f2b1a32fb937314875c0a |
| Phi-3-Medium | 78 | Not recorded | 6a4f2b1a32fb937314875c0a |
| DeepSeek-R1 | 49.1 | Not recorded | 6a4f2b1a32fb937314875c0a |
| Llama-3.1-70B | 46.8 | Not recorded | 6a4f2b1a32fb937314875c0a |

### Gemma-3

DGI: 84.9. Result ID: 6a4f465f60b88bb339e46170.

[Reporting paper](<https://www.sota2.com/research/paper/6a4f2b1a32fb937314875c0a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa779f1e209131cea746cd.md>)

Recorded evaluation configuration: {"Parameters":"27B"}. This record does not assert independent reproduction.

### Phi-4

DGI: 81.4. Result ID: 6a4f465f60b88bb339e46171.

[Reporting paper](<https://www.sota2.com/research/paper/6a4f2b1a32fb937314875c0a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76241e209131cea6b756.md>)

Recorded evaluation configuration: {"Parameters":"14B"}. This record does not assert independent reproduction.

### Mistral-Small-3.1-24B

DGI: 79.5. Result ID: 6a4f465f60b88bb339e46172.

[Reporting paper](<https://www.sota2.com/research/paper/6a4f2b1a32fb937314875c0a.md>)

Recorded evaluation configuration: {"Parameters":"24B"}. This record does not assert independent reproduction.

### Phi-3-Medium

DGI: 78. Result ID: 6a4f465f60b88bb339e46173.

[Reporting paper](<https://www.sota2.com/research/paper/6a4f2b1a32fb937314875c0a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74ae1e209131cea632e1.md>)

Recorded evaluation configuration: {"Parameters":"14B"}. This record does not assert independent reproduction.

### DeepSeek-R1

DGI: 49.1. Result ID: 6a4f465f60b88bb339e46174.

[Reporting paper](<https://www.sota2.com/research/paper/6a4f2b1a32fb937314875c0a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Full Name":"DeepSeek-R1-Distill-Qwen-32B","Parameters":"32B"}. This record does not assert independent reproduction.

### Llama-3.1-70B

DGI: 46.8. Result ID: 6a4f465f60b88bb339e46175.

[Reporting paper](<https://www.sota2.com/research/paper/6a4f2b1a32fb937314875c0a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75d11e209131cea6a028.md>)

[Implementation](<https://github.com/meta-llama/llama-agentic-system>)

Recorded evaluation configuration: {"Parameters":"70B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.