Paper Question Answering on AirQA-Real
285Text ScoreQwen2.5-7B-Instruct (DTFT)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Qwen2.5-7B-Instruct (DTFT)RAG Strategy=NeuSym RAG, Backbone=Qwen2.5-7B-Instruct, Fine-tuning=DTFT2026.01 | 285 | 20 | 100 | 83 | 227 | 255 | 20.2 | |
| Qwen2.5-7B-Instruct (DTFT) + DFPO (PaperGuide)RAG Strategy=NeuSym RAG, Backbone=Qwen2.5-7B-Instruct, Fine-tuning=DTFT, RL Method=DFPO2026.01 | 281 | 40 | 100 | 83 | 182 | 253 | 21 | |
| Qwen2.5-3B-Instruct (DTFT) + DFPO (PaperGuide)RAG Strategy=NeuSym RAG, Backbone=Qwen2.5-3B-Instruct, Fine-tuning=DTFT, RL Method=DFPO2026.01 | 266 | 30 | 100 | 56 | 91 | 237 | 20 | |
| Qwen2.5-7B-Instruct (SFT)RAG Strategy=NeuSym RAG, Backbone=Qwen2.5-7B-Instruct, Fine-tuning=SFT2026.01 | 266 | 10 | 75 | 56 | 318 | 239 | 17.8 | |
| Qwen2.5-3B-Instruct (DTFT)RAG Strategy=NeuSym RAG, Backbone=Qwen2.5-3B-Instruct, Fine-tuning=DTFT2026.01 | 253 | 20 | 75 | 56 | 45 | 226 | 17.6 | |
| Qwen2.5-7B-Instruct (SFT) + DAPORAG Strategy=NeuSym RAG, Backbone=Qwen2.5-7B-Instruct, Fine-tuning=SFT, RL Method=DAPO2026.01 | 253 | 40 | 125 | 83 | 227 | 231 | 17 | |
| Qwen2.5-3B-Instruct (SFT)RAG Strategy=NeuSym RAG, Backbone=Qwen2.5-3B-Instruct, Fine-tuning=SFT2026.01 | 247 | 20 | 75 | 56 | 45 | 222 | 16.7 | |
| Qwen2.5-7B-Instruct (SFT) + M-GRPORAG Strategy=NeuSym RAG, Backbone=Qwen2.5-7B-Instruct, Fine-tuning=SFT, RL Method=M-GRPO2026.01 | 247 | 10 | 0 | 0 | 91 | 212 | 16.1 | |
| Qwen2.5-3B-Instruct (SFT) + DAPORAG Strategy=NeuSym RAG, Backbone=Qwen2.5-3B-Instruct, Fine-tuning=SFT, RL Method=DAPO2026.01 | 243 | 40 | 125 | 28 | 318 | 224 | 17.1 | |
| Qwen2.5-3B-Instruct (SFT) + M-GRPORAG Strategy=NeuSym RAG, Backbone=Qwen2.5-3B-Instruct, Fine-tuning=SFT, RL Method=M-GRPO2026.01 | 223 | 0 | 25 | 28 | 45 | 197 | 13.3 | |
| GPT-4o-miniRAG Strategy=Classic RAG, Training Strategy=Prompting2026.01 | 123 | 119 | 125 | 167 | 136 | 134 | — | |
| DeepSeek-R1RAG Strategy=Classic RAG, Training Strategy=Prompting2026.01 | 117 | 139 | 95 | 306 | 91 | 139 | — | |
| Qwen2.5-VL-72B-InstructRAG Strategy=Classic RAG, Training Strategy=Prompting2026.01 | 96 | 59 | 119 | 111 | 136 | 105 | — | |
| Qwen2.5-7B-InstructRAG Strategy=Classic RAG, Training Strategy=Prompting2026.01 | 81 | 0 | 25 | 28 | 45 | 74 | — | |
| Qwen2.5-3B-InstructRAG Strategy=Classic RAG, Training Strategy=Prompting2026.01 | 68 | 0 | 0 | 28 | 45 | 61 | — |