Literature Retrieval on ScholarGym Fast (test)
95RecallGemini3-Pro
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Gemini3-ProWorkflow=ScholarGym Agentic, Retrieval Type=sparse, Content Scope=Abstract-only, Iterations=5, Extended Thinking=false2026.01 | 95 | 19.9 | 32.9 | 95.8 | 1.1 | 2.2 | |
| DeepSeek-V3.2Workflow=ScholarGym Agentic, Retrieval Type=sparse, Content Scope=Abstract-only, Iterations=5, Extended Thinking=false2026.01 | 85.5 | 13.5 | 23.3 | 86.2 | 0.7 | 1.4 | |
| GPT-5.2Workflow=ScholarGym Agentic, Retrieval Type=sparse, Content Scope=Abstract-only, Iterations=5, Extended Thinking=false2026.01 | 83.7 | 30.5 | 44.7 | 88.3 | 0.9 | 1.8 | |
| DeepSeek-V3.2Workflow=ScholarGym Agentic, Retrieval Type=sparse, Content Scope=Abstract-only, Iterations=5, Extended Thinking=true2026.01 | 81.2 | 28.7 | 42.3 | 87.2 | 0.9 | 1.8 | |
| GLM-4.7Workflow=ScholarGym Agentic, Retrieval Type=sparse, Content Scope=Abstract-only, Iterations=5, Extended Thinking=false2026.01 | 75.4 | 11.1 | 19.4 | 81.4 | 1 | 2 | |
| Qwen3-30BWorkflow=ScholarGym Agentic, Retrieval Type=sparse, Content Scope=Abstract-only, Iterations=5, Extended Thinking=false2026.01 | 67.3 | 18.1 | 28.5 | 72 | 1.1 | 2.1 | |
| Qwen3-8BWorkflow=ScholarGym Agentic, Retrieval Type=sparse, Content Scope=Abstract-only, Iterations=5, Extended Thinking=false2026.01 | 48.3 | 15.2 | 23.1 | 55 | 2.2 | 4.2 | |
| Qwen3-30BWorkflow=ScholarGym Agentic, Retrieval Type=sparse, Content Scope=Abstract-only, Iterations=5, Extended Thinking=true2026.01 | 48.2 | 29 | 36.2 | 59 | 2.5 | 4.8 | |
| Qwen3-8BWorkflow=ScholarGym Agentic, Retrieval Type=sparse, Content Scope=Abstract-only, Iterations=5, Extended Thinking=true2026.01 | 45.8 | 21.6 | 29.3 | 58 | 2.3 | 4.5 | |
| Qwen3-30BWorkflow=Direct Query Baseline, Retrieval Type=sparse, Content Scope=Abstract-only, Iterations=12026.01 | 31.2 | 5.8 | 9.8 | 35 | 5.2 | 9.1 | |
| Qwen3-8BWorkflow=Direct Query Baseline, Retrieval Type=sparse, Content Scope=Abstract-only, Iterations=12026.01 | 18.5 | 4.2 | 6.9 | 21 | 3.6 | 6.1 |