Reasoning on ProntoQA
95AccQUIRE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| QUIREBackbone=Llama3.1-8B2024.05 | 95 | 92.7 | 89.2 | |
| QUIRE (w/o AAE Recall)Backbone=Llama3.1-8B2024.05 | 95 | 87.5 | 84.6 | |
| QUIRE (w/o IG Vote)Backbone=Llama3.1-8B2024.05 | 94.3 | 87 | 83.4 | |
| SCBackbone=Llama3.1-8B2024.05 | 93.2 | 87.5 | 83.6 | |
| QUIREBackbone=Gemma2-9B2024.05 | 92.5 | 71.2 | 69.5 | |
| LtMBackbone=Llama3.1-8B2024.05 | 90 | 77.3 | 72.6 | |
| LtMBackbone=Gemma2-9B2024.05 | 90 | 71 | 67.6 | |
| QUIRE (w/o AAE Recall)Backbone=Gemma2-9B2024.05 | 89 | 64.5 | 61.9 | |
| SRBackbone=Llama3.1-8B2024.05 | 88.5 | 91.5 | 84.5 | |
| CoTBackbone=Llama3.1-8B2024.05 | 86.8 | 86.1 | 78 | |
| SCBackbone=Gemma2-9B2024.05 | 81 | 64.5 | 60.5 | |
| CoTBackbone=Gemma2-9B2024.05 | 77 | 62.7 | 57.7 | |
| QUIRE (w/o IG Vote)Backbone=Gemma2-9B2024.05 | 74.5 | 65.9 | 60.6 | |
| SRBackbone=Gemma2-9B2024.05 | 56.5 | 45.3 | 51.9 |