Probabilistic Forecasting on Metaculus and Polymarket (test)
0.061Brier ScoreHuman Baseline
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Human Baseline2026.05 | 0.061 | 92.3 | 95.8 | 0.055 | — | |
| BBC (binary+human)Input LLM=Claude-Sonnet-42026.05 | 0.125 | 83.7 | 74.2 | 0.027 | 8.775 | |
| BBC (binary only)Input LLM=Claude-Sonnet-42026.05 | 0.128 | 83.3 | 73.2 | 0.036 | 9.004 | |
| Platt ScalingInput LLM=Claude-Sonnet-42026.05 | 0.129 | 82.7 | 72.3 | 0.034 | — | |
| Isotonic RegressionInput LLM=Claude-Sonnet-42026.05 | 0.129 | 83.2 | 72.4 | 0.038 | — | |
| BBC (binary+human)Input LLM=Qwen3-32B2026.05 | 0.133 | 83.3 | 68.6 | 0.046 | 9.402 | |
| BBC (binary+human)Input LLM=Llama-3.3-70B-Instruct2026.05 | 0.135 | 82.9 | 67.9 | 0.045 | 9.526 | |
| BBC (binary only)Input LLM=Qwen3-32B2026.05 | 0.135 | 83.2 | 68.4 | 0.044 | 11.085 | |
| Isotonic RegressionInput LLM=Qwen3-32B2026.05 | 0.137 | 82.3 | 66.2 | 0.04 | — | |
| Future-as-a-label-32BInput LLM=Qwen3-32B2026.05 | 0.137 | 82.9 | 67.7 | 0.046 | — | |
| BBC (binary+human)Input LLM=Qwen3-8B2026.05 | 0.137 | 82.8 | 67.3 | 0.05 | 9.73 | |
| Isotonic RegressionInput LLM=Llama-3.3-70B-Instruct2026.05 | 0.138 | 82.2 | 65.4 | 0.043 | — | |
| BBC (binary only)Input LLM=Llama-3.3-70B-Instruct2026.05 | 0.138 | 81.6 | 67.1 | 0.054 | 11.564 | |
| BBC (binary only)Input LLM=Qwen3-8B2026.05 | 0.138 | 82.4 | 66.2 | 0.044 | 11.55 | |
| Platt ScalingInput LLM=Llama-3.3-70B-Instruct2026.05 | 0.139 | 82.9 | 65.5 | 0.051 | — | |
| Isotonic RegressionInput LLM=Qwen3-8B2026.05 | 0.141 | 81.8 | 63.8 | 0.054 | — | |
| Platt ScalingInput LLM=Qwen3-32B2026.05 | 0.142 | 82.7 | 66.1 | 0.079 | — | |
| Ensemble (n = 3)Input LLM=Claude-Sonnet-42026.05 | 0.143 | 80 | 73.6 | 0.1 | — | |
| Ensemble (n = 10)Input LLM=Qwen3-32B2026.05 | 0.143 | 81.3 | 70 | 0.097 | — | |
| VerbalizedInput LLM=Claude-Sonnet-42026.05 | 0.146 | 79.9 | 72.3 | 0.104 | — | |
| Ensemble (n = 10)Input LLM=Llama-3.3-70B-Instruct2026.05 | 0.151 | 78.2 | 66.9 | 0.099 | — | |
| Platt ScalingInput LLM=Qwen3-8B2026.05 | 0.151 | 82.3 | 63.3 | 0.094 | — | |
| VerbalizedInput LLM=Llama-3.3-70B-Instruct2026.05 | 0.157 | 77.7 | 65.5 | 0.119 | — | |
| OpenForecaster-8BInput LLM=Qwen3-8B2026.05 | 0.157 | 79.4 | 66.3 | 0.084 | — | |
| VerbalizedInput LLM=Qwen3-32B2026.05 | 0.158 | 79.6 | 66.1 | 0.111 | — | |
| Ensemble (n = 10)Input LLM=Qwen3-8B2026.05 | 0.169 | 75.5 | 66.1 | 0.148 | — | |
| VerbalizedInput LLM=Qwen3-8B2026.05 | 0.185 | 75 | 63.3 | 0.164 | — | |
| P(True)Input LLM=Qwen3-8B2026.05 | 0.222 | 77.2 | 61.9 | 0.22 | — | |
| P(True)Input LLM=Qwen3-32B2026.05 | 0.232 | 76.1 | 66.4 | 0.23 | — | |
| P(True)Input LLM=Llama-3.3-70B-Instruct2026.05 | 0.265 | 72.6 | 65.6 | 0.265 | — |