Multi-hop Reasoning on PrOntoQA (Hop Accuracy Breakdown)
97.3Accuracy (1 Hop)GPT-4o
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GPT-4o2025.10 | 97.3 | 74.4 | 66.4 | |
| Llama3.1 70B itModel Backbone=Llama3.1 70B, Type=Instruction-tuned2025.10 | 96.2 | 66.7 | 62.1 | |
| ARModel Backbone=Llama3.1 8B2025.10 | 95 | 95.6 | 95.3 | |
| ARModel Backbone=Gemma2 9B2025.10 | 93.5 | 93.5 | 93.5 | |
| Gemma2 27B itModel Backbone=Gemma2 27B, Type=Instruction-tuned2025.10 | 91.3 | 77.6 | 73.9 | |
| DeepSeek-R1-8BModel Backbone=DeepSeek-R1-8B2025.10 | 80 | 76 | 64.4 | |
| Llama3.1 8BModel Backbone=Llama3.1 8B2025.10 | 51 | 50.8 | 50.3 | |
| Gemma2 9BModel Backbone=Gemma2 9B2025.10 | 48.5 | 47.5 | 47.9 |