Binary decision task on ISD India (test)
78.1AccuracyOG-MAR
Evaluation results
| Method | Links | |
|---|---|---|
| OG-MARBackbone=EXAONE 3.52026.01 | 78.1 | |
| RoleBackbone=QWEN 2.52026.01 | 74.52 | |
| Zero-shotBackbone=QWEN 2.52026.01 | 72.22 | |
| OG-MARBackbone=Gemini 2.5 Flash Lite2026.01 | 70.07 | |
| DebateBackbone=EXAONE 3.52026.01 | 67.73 | |
| OG-MARBackbone=QWEN 2.52026.01 | 65.99 | |
| ValuesRAGBackbone=QWEN 2.52026.01 | 65.91 | |
| DebateBackbone=GPT-4o mini2026.01 | 65.68 | |
| ValuesRAGBackbone=EXAONE 3.52026.01 | 64.46 | |
| ValuesRAGBackbone=GPT-4o mini2026.01 | 64.2 | |
| DebateBackbone=Gemini 2.5 Flash Lite2026.01 | 63.35 | |
| DebateBackbone=QWEN 2.52026.01 | 63.2 | |
| Self-consistencyBackbone=QWEN 2.52026.01 | 62.78 | |
| OG-MARBackbone=GPT-4o mini2026.01 | 61.92 | |
| Zero-shotBackbone=GPT-4o mini2026.01 | 61.39 | |
| Self-consistencyBackbone=EXAONE 3.52026.01 | 61.22 | |
| RoleBackbone=GPT-4o mini2026.01 | 60.6 | |
| RoleBackbone=EXAONE 3.52026.01 | 60.48 | |
| ValuesRAGBackbone=Gemini 2.5 Flash Lite2026.01 | 60.41 | |
| Zero-shotBackbone=EXAONE 3.52026.01 | 60.41 | |
| Self-consistencyBackbone=GPT-4o mini2026.01 | 59.76 | |
| RoleBackbone=Gemini 2.5 Flash Lite2026.01 | 55.21 | |
| Zero-shotBackbone=Gemini 2.5 Flash Lite2026.01 | 50 | |
| Self-consistencyBackbone=Gemini 2.5 Flash Lite2026.01 | 47.05 |