Future-aligned research proposal prediction on future corpus of ML papers (test)
71.4Hypothesis ScoreFuture-aligned SFT + Stepwise CoT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Future-aligned SFT + Stepwise CoTBackbone=Qwen2.5-14B-Instruct2026.03 | 71.4 | 63.5 | 61.8 | 56.7 | 69.7 | |
| Future-aligned SFT + Stepwise CoTBackbone=Qwen2.5-7B-Instruct2026.03 | 68.7 | 60.5 | 59.1 | 54.9 | 66.5 | |
| Future-aligned SFT (w/o stepwise)Backbone=Qwen2.5-14B-Instruct2026.03 | 68.7 | 60 | 58.2 | 56.8 | 66.1 | |
| Future-aligned SFT (w/o stepwise)Backbone=Qwen2.5-7B-Instruct2026.03 | 68.3 | 60.2 | 59.2 | 54.8 | 65.9 | |
| Future-aligned SFT + Stepwise CoTBackbone=Llama-3.1-8B-Instruct2026.03 | 68.1 | 61 | 58.7 | 51.9 | 65.3 | |
| Future-aligned SFT (w/o reasoning traces)Backbone=Qwen2.5-14B-Instruct2026.03 | 68 | 60 | 58.9 | 55.1 | 65.1 | |
| Future-aligned SFT (w/o reasoning traces)Backbone=Qwen2.5-7B-Instruct2026.03 | 67.9 | 60 | 58.7 | 54.9 | 65.3 | |
| Future-aligned SFT (w/o stepwise)Backbone=Llama-3.1-8B-Instruct2026.03 | 67.2 | 59.2 | 57.6 | 55.1 | 65 | |
| Future-aligned SFT (w/o reasoning traces)Backbone=Llama-3.1-8B-Instruct2026.03 | 66.7 | 58.3 | 56.9 | 54.6 | 64.1 | |
| RQ onlyBackbone=Qwen2.5-7B-Instruct2026.03 | 66.7 | 56.6 | 55.2 | 52 | 63.3 | |
| PromptingBackbone=Qwen2.5-7B-Instruct2026.03 | 66.3 | 57.8 | 55.1 | 54.8 | 63.6 | |
| RQ onlyBackbone=Qwen2.5-14B-Instruct2026.03 | 65.8 | 56.4 | 54.4 | 53.6 | 62.6 | |
| PromptingBackbone=Qwen2.5-14B-Instruct2026.03 | 65.8 | 57.1 | 55.1 | 55.6 | 63 | |
| PromptingBackbone=Llama-3.1-8B-Instruct2026.03 | 64.5 | 56.8 | 54.4 | 55 | 62.1 | |
| Chain-of-IdeasBackbone=Qwen2.5-14B-Instruct2026.03 | 63.8 | 54.7 | 52.1 | 46.8 | 60.8 | |
| Chain-of-IdeasBackbone=Llama-3.1-8B-Instruct2026.03 | 63.2 | 54.1 | 51.6 | 45.9 | 59.3 | |
| RQ onlyBackbone=Llama-3.1-8B-Instruct2026.03 | 63 | 52.8 | 51.1 | 52.4 | 60 | |
| Chain-of-IdeasBackbone=Qwen2.5-7B-Instruct2026.03 | 62 | 53.2 | 50.3 | 47.1 | 59.3 | |
| AI-ResearcherBackbone=Qwen2.5-14B-Instruct2026.03 | 58.5 | 48.1 | 46 | 45 | 55.3 | |
| AI-ResearcherBackbone=Qwen2.5-7B-Instruct2026.03 | 58 | 47.5 | 45.6 | 44.8 | 54.1 | |
| AI-ResearcherBackbone=Llama-3.1-8B-Instruct2026.03 | 57.9 | 46 | 45.1 | 44.4 | 53.7 | |
| Paper onlyBackbone=Qwen2.5-7B-Instruct2026.03 | 56.4 | 50.5 | 46.6 | 47.2 | 52.9 | |
| Paper onlyBackbone=Llama-3.1-8B-Instruct2026.03 | 55.2 | 49.4 | 46.9 | 48 | 52.4 | |
| Paper onlyBackbone=Qwen2.5-14B-Instruct2026.03 | 54.3 | 49.3 | 45.6 | 46.6 | 51.3 |