Geopolitical Event Forecasting on POLECAT-FOR-ME Llama 3.1 knowledge cut-off compliant (test)
22.2ISRGPT-4o
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-4oType=Proprietary LLM2026.01 | 22.2 | 17.3 | 15.3 | 37.9 | 18.6 | 15.6 | 20.8 | 15.8 | 23.5 | 24 | |
| GPT-4o-miniType=Proprietary LLM2026.01 | 20.3 | 21.3 | 15.3 | 37.2 | 17.7 | 16.3 | 19.2 | 14.1 | 22.7 | 22.5 | |
| ThinkTank-ME (Elite Ensemble - Weighted Best-of-N)Backbone=Llama-3.1-8B, Leader Strategy=Elite Ensemble, Variant=Weighted Best-of-N2026.01 | 18.3 | 15.7 | 14.4 | 51.6 | 23.4 | 26.9 | 11.5 | 17.5 | 24.6 | 25 | |
| ThinkTank-ME (Wisdom Aggregation - Weighted Best-of-N)Backbone=Llama-3.1-8B, Leader Strategy=Wisdom Aggregation, Variant=Weighted Best-of-N2026.01 | 18 | 13.9 | 14.4 | 50.7 | 22.2 | 26.6 | 10.9 | 17.5 | 23.9 | 24.4 | |
| ThinkTank-ME (Elite Ensemble - Majority Voting)Backbone=Llama-3.1-8B, Leader Strategy=Elite Ensemble, Variant=Majority Voting2026.01 | 18 | 12.8 | 13.5 | 50.7 | 22.6 | 26.9 | 11.2 | 17 | 23.9 | 24.2 | |
| ThinkTank-ME (Wisdom Aggregation - Majority Voting)Backbone=Llama-3.1-8B, Leader Strategy=Wisdom Aggregation, Variant=Majority Voting2026.01 | 17.7 | 11.9 | 13.9 | 50.4 | 21.6 | 26.9 | 11.2 | 18.1 | 23.6 | 23.9 | |
| ThinkTank-ME (Elite Ensemble - Vanilla Best-of-N)Backbone=Llama-3.1-8B, Leader Strategy=Elite Ensemble, Variant=Vanilla Best-of-N2026.01 | 17.7 | 14.1 | 11.8 | 50.9 | 19.2 | 24.8 | 8 | 16.1 | 22.8 | 23.9 | |
| Baselines (All-data Training)Training=All-data2026.01 | 17.4 | 10.7 | 16.2 | 49.9 | 19.3 | 25.3 | 8.5 | 15.8 | 22.7 | 22.6 | |
| ThinkTank-ME (Wisdom Aggregation - Vanilla Best-of-N)Backbone=Llama-3.1-8B, Leader Strategy=Wisdom Aggregation, Variant=Vanilla Best-of-N2026.01 | 16.3 | 13.1 | 9.6 | 48.4 | 18.2 | 19.6 | 9 | 15.5 | 21.2 | 22.2 | |
| ThinkTank-ME (Expert Routing)Backbone=Llama-3.1-8B, Leader Strategy=Expert Routing2026.01 | 13.9 | 8.4 | 11.4 | 39 | 15.6 | 23.7 | 10.7 | 13.5 | 18.6 | 19.4 | |
| Baselines (No Training)Training=None2026.01 | 11.1 | 7.8 | 9.6 | 34 | 17.8 | 12.4 | 10.1 | 8.3 | 15.5 | 15.7 |