Mathematical Reasoning on GSM8K (Answer Accuracy, Mode Accuracy, & Generation Count)
97.2Answer AccuracyL = 2 (known prior)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| L = 2 (known prior)Model=Qwen-2.5-72B, 1 - delta=0.9752026.02 | 97.2 | 99.8 | 2.19 | |
| L = 2 (known prior)Model=Qwen-2.5-72B, 1 - delta=0.992026.02 | 97.2 | 99.8 | 2.51 | |
| L = 3 (known prior)Model=Qwen-2.5-72B, 1 - delta=0.9752026.02 | 97.2 | 99.8 | 2.15 | |
| L = 3 (known prior)Model=Qwen-2.5-72B, 1 - delta=0.992026.02 | 97.2 | 99.8 | 2.44 | |
| L = 2 (uncertain prior)Model=Qwen-2.5-72B, 1 - delta=0.992026.02 | 97.2 | 100 | 3.37 | |
| L = 3 (uncertain prior)Model=Qwen-2.5-72B, 1 - delta=0.992026.02 | 97.2 | 100 | 2.95 | |
| ASCModel=Qwen-2.5-72B, 1 - delta=0.952026.02 | 97.2 | 100 | 4.58 | |
| ASCModel=Qwen-2.5-72B, 1 - delta=0.9752026.02 | 97.2 | 100 | 5.61 | |
| ASCModel=Qwen-2.5-72B, 1 - delta=0.992026.02 | 97.2 | 100 | 6.91 | |
| L = 2 (known prior)Model=LLaMA-3.1-405B, 1 - delta=0.992026.02 | 97.2 | 100 | 2.31 | |
| L = 3 (known prior)Model=LLaMA-3.1-405B, 1 - delta=0.992026.02 | 97.2 | 100 | 2.25 | |
| L = 2 (uncertain prior)Model=LLaMA-3.1-405B, 1 - delta=0.9752026.02 | 97.2 | 99.7 | 2.49 | |
| L = 2 (uncertain prior)Model=LLaMA-3.1-405B, 1 - delta=0.992026.02 | 97.2 | 99.7 | 2.7 | |
| L = 3 (uncertain prior)Model=LLaMA-3.1-405B, 1 - delta=0.9752026.02 | 97.2 | 99.7 | 2.42 | |
| L = 3 (uncertain prior)Model=LLaMA-3.1-405B, 1 - delta=0.992026.02 | 97.2 | 99.7 | 2.64 | |
| ASCModel=LLaMA-3.1-405B, 1 - delta=0.952026.02 | 97.2 | 100 | 4.5 | |
| ASCModel=LLaMA-3.1-405B, 1 - delta=0.9752026.02 | 97.2 | 100 | 5.78 | |
| ASCModel=LLaMA-3.1-405B, 1 - delta=0.992026.02 | 97.2 | 100 | 7.02 | |
| L = 2 (known prior)Model=Qwen-2.5-72B, 1 - delta=0.952026.02 | 97.1 | 99.8 | 2.03 | |
| L = 3 (known prior)Model=Qwen-2.5-72B, 1 - delta=0.952026.02 | 97.1 | 99.7 | 2 | |
| L = 3 (known prior)Model=LLaMA-3.1-405B, 1 - delta=0.9752026.02 | 97.1 | 99.9 | 2.04 | |
| L = 2 (known prior)Model=LLaMA-3.1-405B, 1 - delta=0.952026.02 | 97 | 99.8 | 1.95 | |
| L = 2 (known prior)Model=LLaMA-3.1-405B, 1 - delta=0.9752026.02 | 97 | 99.8 | 2.09 | |
| L = 3 (known prior)Model=LLaMA-3.1-405B, 1 - delta=0.952026.02 | 97 | 99.8 | 1.88 | |
| ASCModel=GPT-4o mini, 1 - delta=0.952026.02 | 96.8 | 99.6 | 5.33 | |
| ASCModel=GPT-4o mini, 1 - delta=0.9752026.02 | 96.8 | 99.6 | 6.64 | |
| L = 3 (known prior)Model=GPT-4o mini, 1 - delta=0.9752026.02 | 96.7 | 99.5 | 3.73 | |
| L = 2 (known prior)Model=GPT-4o mini, 1 - delta=0.952026.02 | 96.6 | 99.4 | 3.53 | |
| L = 2 (known prior)Model=GPT-4o mini, 1 - delta=0.9752026.02 | 96.6 | 99.4 | 3.81 | |
| L = 3 (known prior)Model=GPT-4o mini, 1 - delta=0.952026.02 | 96.6 | 99.4 | 3.42 | |
| L = 2 (uncertain prior)Model=GPT-4o mini, 1 - delta=0.9752026.02 | 96.6 | 99.1 | 3.98 | |
| L = 2 (uncertain prior)Model=GPT-4o mini, 1 - delta=0.992026.02 | 96.6 | 99 | 4.41 | |
| L = 3 (uncertain prior)Model=GPT-4o mini, 1 - delta=0.9752026.02 | 96.6 | 99 | 3.49 | |
| L = 3 (uncertain prior)Model=GPT-4o mini, 1 - delta=0.992026.02 | 96.6 | 99 | 3.8 | |
| L = 2 (known prior)Model=GPT-4o mini, 1 - delta=0.992026.02 | 96.4 | 99.7 | 4.04 | |
| L = 3 (known prior)Model=GPT-4o mini, 1 - delta=0.992026.02 | 96.4 | 99.7 | 3.94 | |
| ASCModel=GPT-4o mini, 1 - delta=0.992026.02 | 96.4 | 100 | 8.27 | |
| L = 2 (uncertain prior)Model=Qwen-2.5-72B, 1 - delta=0.952026.02 | 96.3 | 98.2 | 1 | |
| L = 2 (uncertain prior)Model=Qwen-2.5-72B, 1 - delta=0.9752026.02 | 96.3 | 98.2 | 1 | |
| L = 3 (uncertain prior)Model=Qwen-2.5-72B, 1 - delta=0.952026.02 | 96.3 | 98.2 | 1 | |
| L = 3 (uncertain prior)Model=Qwen-2.5-72B, 1 - delta=0.9752026.02 | 96.3 | 98.2 | 1 | |
| L = 2 (uncertain prior)Model=LLaMA-3.1-405B, 1 - delta=0.952026.02 | 96.2 | 97.7 | 1 | |
| L = 3 (uncertain prior)Model=LLaMA-3.1-405B, 1 - delta=0.952026.02 | 96.2 | 97.7 | 1 | |
| L = 2 (uncertain prior)Model=GPT-4o mini, 1 - delta=0.952026.02 | 94.3 | 96.5 | 1 | |
| L = 3 (uncertain prior)Model=GPT-4o mini, 1 - delta=0.952026.02 | 94.3 | 96.5 | 1 |