Question Answering on ConFiQA-QA counterfactual contexts
81.2AccuracyRAAT + DTA
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| RAAT + DTAStrategy=Divide-Then-Align2025.05 | 81.2 | 84.6 | 78.1 | 81.2 | 77 | 85.6 | 81.1 | |
| RAAT + LogitsStrategy=Abstention via Logits2025.05 | 47.9 | 74.3 | 46.2 | 56.9 | 16 | 60.5 | 25.3 | |
| RAAT + ICLStrategy=In-Context Learning, Examples=32025.05 | 44.6 | 80.1 | 44.1 | 56.8 | 1.76 | 100 | 3.45 | |
| RAATStrategy=Baseline2025.05 | 43.5 | 79.5 | 43.5 | 56.2 | 0 | 0 | 0 | |
| RAAT + ConsistencyStrategy=Abstention via Self-Consistency, Number of responses (n)=102025.05 | 42.1 | 75 | 41.8 | 53.6 | 2.53 | 57.5 | 4.86 | |
| RAAT + P(True)Strategy=Abstention via P(True)2025.05 | 41.5 | 63.7 | 41.1 | 49.9 | 14.6 | 43.3 | 21.8 | |
| Llama-2-7BBackbone=Llama-2-7B2025.05 | 41.4 | 75.8 | 41.4 | 53.5 | 0 | 0 | 0 |