Scientific Method Comparative Reasoning Forecasting on Independently Constructed 2025 (test)
67.49AccuracyReason-SFT-DrGRPO
Evaluation results
| Method | Links | |
|---|---|---|
| Reason-SFT-DrGRPOBackbone=Qwen3, Evaluation Protocol=Zero-shot2026.04 | 67.49 | |
| Reason-DAPOBackbone=Qwen3, Evaluation Protocol=Zero-shot2026.04 | 65.94 | |
| Direct-SFTBackbone=Qwen3, Evaluation Protocol=Zero-shot2026.04 | 63.43 | |
| Reason-SFT-DAPOBackbone=Qwen3, Evaluation Protocol=Zero-shot2026.04 | 61.83 | |
| Synthetic-Reason-SFT-DAPOBackbone=Qwen3, Evaluation Protocol=Zero-shot2026.04 | 56.46 | |
| GPT-4.1Retrieval Augmentation=true, Evaluation Protocol=Zero-shot2026.04 | 51.4 | |
| Direct-SFTBackbone=Llama3.1, Evaluation Protocol=Zero-shot2026.04 | 41.94 | |
| Base (Reasoning)Backbone=Llama3.1, Reasoning Strategy=CoT instruction, Evaluation Protocol=Zero-shot2026.04 | 36.29 | |
| Base (Reasoning)Backbone=Qwen3, Reasoning Strategy=Thinking variant, Evaluation Protocol=Zero-shot2026.04 | 20.06 | |
| BaseBackbone=Llama3.1, Reasoning Strategy=Non-thinking, Evaluation Protocol=Zero-shot2026.04 | 12.8 | |
| BaseBackbone=Qwen3, Reasoning Strategy=Non-thinking, Evaluation Protocol=Zero-shot2026.04 | 2.69 |