Speaker-attribute classification on WhoSaidIt re-annotated (test)
91.9Male AccuracyGPT 4.1
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| GPT 4.12026.05 | 91.9 | 94.1 | 69 | 86.5 | 91.5 | 92.9 | 91.7 | 85.7 | 75.3 | |
| Claude 3.7 Sonnet2026.05 | 91.9 | 96.5 | 70 | 86.8 | 94.5 | 92.4 | 90.3 | 90.9 | 78.6 | |
| Gemini 2.5 Flash2026.05 | 90.9 | 88.3 | 66.5 | 81.6 | 81.2 | 83.9 | 90.1 | 92.8 | 69.5 | |
| DeepSeek V32026.05 | 86.2 | 82.3 | 66.9 | 84.1 | 84.5 | 88.1 | 89.9 | 78.2 | 74.1 | |
| GPT 4.1prompting=with rationales2026.05 | 85 | 88.9 | 42.5 | 83 | 83.5 | 86.6 | 85.7 | 72.9 | 63.2 | |
| Claude 3.7 Sonnetprompting=with rationales2026.05 | 84.8 | 93.3 | 43.9 | 83.3 | 89.4 | 85.5 | 83.4 | 83 | 66.1 | |
| Gemini 2.5 Flashprompting=with rationales2026.05 | 83.2 | 77.9 | 40.4 | 77.5 | 64 | 70.1 | 83 | 86.6 | 57.9 | |
| DeepSeek V3prompting=with rationales2026.05 | 74.7 | 67 | 38.5 | 79.4 | 70.2 | 77.7 | 82.5 | 58.6 | 61.7 |