Survey Generation on Scientific Survey Generation (test)
0.717PrecisionOutlineForge
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| OutlineForgeBackbone=Phi-3.8B, Fine-tuned=true, Inference Steps=3002026.01 | 0.717 | 0.311 | 0.381 | |
| OutlineForgeBackbone=Phi-3.8B, Fine-tuned=true, Inference Steps=2002026.01 | 0.678 | 0.317 | 0.422 | |
| OutlineForgeBackbone=GPT-4o-mini, Inference Steps=3002026.01 | 0.668 | 0.312 | 0.397 | |
| OutlineForgeBackbone=GPT-4o-mini, Inference Steps=2002026.01 | 0.612 | 0.264 | 0.352 | |
| OutlineForgeBackbone=Claude-3.5-Haiku, Inference Steps=2002026.01 | 0.533 | 0.173 | 0.249 | |
| OutlineForgeBackbone=Claude-3.5-Haiku, Inference Steps=3002026.01 | 0.524 | 0.186 | 0.246 | |
| OutlineForgeBackbone=Llama-3.1-Instruct-70B, Inference Steps=3002026.01 | 0.512 | 0.196 | 0.21 | |
| OutlineForgeBackbone=Llama-3.1-Instruct-70B, Inference Steps=2002026.01 | 0.446 | 0.161 | 0.223 | |
| OutlineForgeBackbone=Gemma2-2B, Fine-tuned=true, Inference Steps=3002026.01 | 0.406 | 0.251 | 0.247 | |
| OutlineForgeBackbone=Gemma2-2B, Fine-tuned=true, Inference Steps=2002026.01 | 0.399 | 0.166 | 0.201 | |
| SurveyForgeBackbone=Phi-3.8B, Inference Steps=2002026.01 | 0.359 | 0.345 | 0.313 | |
| SurveyForgeBackbone=GPT-4o-mini, Inference Steps=2002026.01 | 0.346 | 0.261 | 0.285 | |
| OutlineForgeBackbone=Qwen3-1.8B, Fine-tuned=true, Inference Steps=3002026.01 | 0.298 | 0.074 | 0.111 | |
| AutoSurveyBackbone=Phi-3.8B, Inference Steps=2002026.01 | 0.296 | 0.306 | 0.274 | |
| AutoSurveyBackbone=GPT-4o-mini, Inference Steps=2002026.01 | 0.295 | 0.181 | 0.213 | |
| SurveyForgeBackbone=Claude-3.5-Haiku, Inference Steps=2002026.01 | 0.267 | 0.265 | 0.243 | |
| OutlineForgeBackbone=Qwen3-1.8B, Fine-tuned=true, Inference Steps=2002026.01 | 0.244 | 0.095 | 0.12 | |
| AutoSurveyBackbone=Claude-3.5-Haiku, Inference Steps=2002026.01 | 0.228 | 0.172 | 0.187 |