Question Answering on DisambiguationQA
89.1Answer AccuracyL-aggregated stopping policy (L=2, known prior)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| L-aggregated stopping policy (L=2, known prior)Model=Qwen-2.5-72B, 1 - delta=0.9752026.02 | 89.1 | 98.4 | 5.54 | |
| L-aggregated stopping policy (L=3, known prior)Model=Qwen-2.5-72B, 1 - delta=0.9752026.02 | 89.1 | 98.4 | 5.48 | |
| L-aggregated stopping policy (L=2, known prior)Model=Qwen-2.5-72B, 1 - delta=0.992026.02 | 89.1 | 98.4 | 6.11 | |
| L-aggregated stopping policy (L=3, known prior)Model=Qwen-2.5-72B, 1 - delta=0.992026.02 | 89.1 | 98.4 | 6.09 | |
| L-aggregated stopping policy (L=2, known prior)Model=Qwen-2.5-72B, 1 - delta=0.952026.02 | 88.5 | 97.9 | 4.93 | |
| L-aggregated stopping policy (L=3, known prior)Model=Qwen-2.5-72B, 1 - delta=0.952026.02 | 88.5 | 97.9 | 4.93 | |
| L-aggregated stopping policy (L=2, uncertain prior)Model=Qwen-2.5-72B, 1 - delta=0.992026.02 | 88.3 | 99.2 | 7.37 | |
| L-aggregated stopping policy (L=3, uncertain prior)Model=Qwen-2.5-72B, 1 - delta=0.992026.02 | 88.3 | 99.2 | 7.44 | |
| ASCModel=Qwen-2.5-72B, 1 - delta=0.952026.02 | 88 | 100 | 6.19 | |
| ASCModel=Qwen-2.5-72B, 1 - delta=0.9752026.02 | 88 | 100 | 7.67 | |
| ASCModel=Qwen-2.5-72B, 1 - delta=0.992026.02 | 88 | 100 | 9.64 | |
| L-aggregated stopping policy (L=2, uncertain prior)Model=Qwen-2.5-72B, 1 - delta=0.952026.02 | 87.7 | 97.6 | 3.45 | |
| L-aggregated stopping policy (L=3, uncertain prior)Model=Qwen-2.5-72B, 1 - delta=0.952026.02 | 87.7 | 97.6 | 3.41 | |
| L-aggregated stopping policy (L=2, uncertain prior)Model=Qwen-2.5-72B, 1 - delta=0.9752026.02 | 87.7 | 97.6 | 4.23 | |
| L-aggregated stopping policy (L=3, uncertain prior)Model=Qwen-2.5-72B, 1 - delta=0.9752026.02 | 87.7 | 97.6 | 4.19 | |
| ASCModel=GPT-4o mini, 1 - delta=0.952026.02 | 78.7 | 100 | 8.05 | |
| ASCModel=GPT-4o mini, 1 - delta=0.9752026.02 | 78.7 | 100 | 10.24 | |
| ASCModel=GPT-4o mini, 1 - delta=0.992026.02 | 78.7 | 100 | 12.56 | |
| L-aggregated stopping policy (L=2, known prior)Model=GPT-4o mini, 1 - delta=0.9752026.02 | 78.1 | 99.5 | 8.9 | |
| L-aggregated stopping policy (L=3, known prior)Model=GPT-4o mini, 1 - delta=0.9752026.02 | 78.1 | 99.5 | 8.9 | |
| L-aggregated stopping policy (L=2, known prior)Model=GPT-4o mini, 1 - delta=0.992026.02 | 78.1 | 99.5 | 10.55 | |
| L-aggregated stopping policy (L=3, known prior)Model=GPT-4o mini, 1 - delta=0.992026.02 | 78.1 | 99.5 | 10.55 | |
| L-aggregated stopping policy (L=2, known prior)Model=GPT-4o mini, 1 - delta=0.952026.02 | 77.9 | 98.7 | 7.65 | |
| L-aggregated stopping policy (L=3, known prior)Model=GPT-4o mini, 1 - delta=0.952026.02 | 77.9 | 98.7 | 7.65 | |
| L-aggregated stopping policy (L=2, uncertain prior)Model=GPT-4o mini, 1 - delta=0.992026.02 | 77.9 | 99.2 | 14.14 | |
| L-aggregated stopping policy (L=3, uncertain prior)Model=GPT-4o mini, 1 - delta=0.992026.02 | 77.9 | 99.2 | 14.21 | |
| L-aggregated stopping policy (L=2, uncertain prior)Model=GPT-4o mini, 1 - delta=0.952026.02 | 77.3 | 97.6 | 7.24 | |
| L-aggregated stopping policy (L=3, uncertain prior)Model=GPT-4o mini, 1 - delta=0.952026.02 | 77.3 | 97.6 | 7.33 | |
| L-aggregated stopping policy (L=2, uncertain prior)Model=GPT-4o mini, 1 - delta=0.9752026.02 | 77.1 | 98.4 | 9.46 | |
| L-aggregated stopping policy (L=3, uncertain prior)Model=GPT-4o mini, 1 - delta=0.9752026.02 | 77.1 | 98.4 | 9.48 | |
| L-aggregated stopping policy (L=3, uncertain prior)Model=LLaMA-3.1-405B, 1 - delta=0.952026.02 | 76.8 | 97.6 | 3.35 | |
| L-aggregated stopping policy (L=3, uncertain prior)Model=LLaMA-3.1-405B, 1 - delta=0.9752026.02 | 76.8 | 97.6 | 3.5 | |
| L-aggregated stopping policy (L=2, uncertain prior)Model=LLaMA-3.1-405B, 1 - delta=0.952026.02 | 76.5 | 97.9 | 3.69 | |
| L-aggregated stopping policy (L=2, uncertain prior)Model=LLaMA-3.1-405B, 1 - delta=0.9752026.02 | 76.5 | 97.9 | 4.06 | |
| L-aggregated stopping policy (L=2, uncertain prior)Model=LLaMA-3.1-405B, 1 - delta=0.992026.02 | 76.5 | 98.9 | 6.76 | |
| L-aggregated stopping policy (L=3, uncertain prior)Model=LLaMA-3.1-405B, 1 - delta=0.992026.02 | 76.5 | 98.9 | 6.59 | |
| L-aggregated stopping policy (L=2, known prior)Model=LLaMA-3.1-405B, 1 - delta=0.9752026.02 | 76.3 | 99.2 | 4.78 | |
| L-aggregated stopping policy (L=3, known prior)Model=LLaMA-3.1-405B, 1 - delta=0.9752026.02 | 76.3 | 99.2 | 4.74 | |
| L-aggregated stopping policy (L=2, known prior)Model=LLaMA-3.1-405B, 1 - delta=0.992026.02 | 76.3 | 99.2 | 5.68 | |
| L-aggregated stopping policy (L=3, known prior)Model=LLaMA-3.1-405B, 1 - delta=0.992026.02 | 76.3 | 99.2 | 5.63 | |
| L-aggregated stopping policy (L=3, known prior)Model=LLaMA-3.1-405B, 1 - delta=0.952026.02 | 76 | 97.3 | 3.6 | |
| ASCModel=LLaMA-3.1-405B, 1 - delta=0.952026.02 | 76 | 100 | 7.49 | |
| ASCModel=LLaMA-3.1-405B, 1 - delta=0.9752026.02 | 76 | 100 | 9.49 | |
| ASCModel=LLaMA-3.1-405B, 1 - delta=0.992026.02 | 76 | 100 | 11.83 | |
| L-aggregated stopping policy (L=2, known prior)Model=LLaMA-3.1-405B, 1 - delta=0.952026.02 | 75.7 | 97.6 | 3.87 |