Identifying relative strengths on AES Feedback Dataset Positive Feedback (test)
35.59Intent ScoreOR
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ORDescription=Operational Rater (Human Baseline)2026.05 | 35.59 | 41.58 | 34.75 | 30.23 | 29.75 | 33.06 | 29.51 | 38.23 | 32.81 | 40.24 | 34.57 | |
| Llama 3.1Backbone=70B, Assessment Formulation=self-referential2026.05 | 22.84 | 70 | 28.19 | 22.38 | 23.74 | 40.52 | 35.71 | 37.64 | 26.89 | 47.7 | 35.56 | |
| GPT-4.1Assessment Formulation=self-referential2026.05 | 21.02 | 74.07 | 29.96 | 19.44 | 23.65 | 21.18 | 37.74 | 25.68 | 29.25 | 37.68 | 31.96 | |
| Qwen 2.5Backbone=72B, Assessment Formulation=self-referential2026.05 | 13.8 | 65.22 | 33.61 | 18.69 | 14.11 | 37.63 | 39.47 | 27.23 | 29.71 | 38.98 | 31.84 |