Scholarly Answer Generation on Scholarly Humanities Papers 100-paper sample
4.46GPT-5.4 Answer AccuracySPIRE (full)
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SPIRE (full)configuration=full seven-agent2026.05 | 4.46 | 4.63 | 4.76 | 4.5 | 4.95 | 4.96 | 4.96 | 4.9 | 4.46 | 3.89 | 4.39 | 3.99 | 4.94 | 4.57 | 4.62 | 4.76 | |
| Naive LLM2026.05 | 3.72 | 2.81 | 3.65 | 2.26 | 4.01 | 2.74 | 3.58 | 2.46 | 3.62 | 2.96 | 2.51 | 2.26 | 4.25 | 2.06 | 2.24 | 1.47 | |
| GraphRAG2026.05 | 2.35 | 2.37 | 2.61 | 1.68 | 2.55 | 1.93 | 2.5 | 1.39 | 2.3 | 2.08 | 1.85 | 1.43 | 2.66 | 1.4 | 1.66 | 1 | |
| Text RAG2026.05 | 1.78 | 1.46 | 1.57 | 1.55 | 1.65 | 1.25 | 1.45 | 1.3 | 1.73 | 1.17 | 1 | 1.3 | 1.76 | 1 | 1 | 1 |