Scholarly Question Answering on SQA CS V2
89.7Overall Scoregemini-2.5-pro
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| gemini-2.5-prointent-aware=true2026.03 | 89.7 | 82.6 | 94.5 | 95.7 | 86.1 | |
| Claude opus-4intent-aware=true2026.03 | 89 | 85.5 | 89.3 | 95.1 | 86 | |
| gemini-2.5-prointent-aware=false2026.03 | 88.1 | 82.6 | 94.1 | 93.2 | 82.4 | |
| o3intent-aware=true2026.03 | 86 | 90.7 | 96.6 | 89.9 | 66.9 | |
| Claude opus-4intent-aware=false2026.03 | 85.4 | 84.3 | 87.9 | 89.6 | 79.6 | |
| o3intent-aware=false2026.03 | 85.1 | 91.4 | 96.5 | 89.4 | 63.4 |