# Paper Question Answering on AirQA-Real Source: [Paper Question Answering on AirQA-Real]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:06:22.195Z Evaluation of model performance on the AirQA-Real dataset for Paper Question Answering, measuring component-specific scores for text, tables, images, and forms. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Paper Question Answering | | Dataset (including any stated split) | AirQA-Real | | Metric | Text Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 15 of 15 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Text Score](): Higher is better; unit: PERCENT. - [Table Score](): Higher is better; unit: PERCENT. - [Image Score](): Higher is better; unit: PERCENT. - [Form Score](): Higher is better; unit: PERCENT. - [Meta Score](): Higher is better; unit: PERCENT. - [Average Score](): Higher is better; unit: PERCENT. - [Intermediate Average Score](): Higher is better; unit: SCALAR. ## Recorded results | Method | Text Score | Result date | Report paper | | --- | --- | --- | --- | | Qwen2.5-7B-Instruct (DTFT) | 285 | Not recorded | 69705d9ae25ad2fa9d1ed223 | | Qwen2.5-7B-Instruct (DTFT) + DFPO (PaperGuide) | 281 | Not recorded | 69705d9ae25ad2fa9d1ed223 | | Qwen2.5-3B-Instruct (DTFT) + DFPO (PaperGuide) | 266 | Not recorded | 69705d9ae25ad2fa9d1ed223 | | Qwen2.5-7B-Instruct (SFT) | 266 | Not recorded | 69705d9ae25ad2fa9d1ed223 | | Qwen2.5-3B-Instruct (DTFT) | 253 | Not recorded | 69705d9ae25ad2fa9d1ed223 | | Qwen2.5-7B-Instruct (SFT) + DAPO | 253 | Not recorded | 69705d9ae25ad2fa9d1ed223 | | Qwen2.5-3B-Instruct (SFT) | 247 | Not recorded | 69705d9ae25ad2fa9d1ed223 | | Qwen2.5-7B-Instruct (SFT) + M-GRPO | 247 | Not recorded | 69705d9ae25ad2fa9d1ed223 | | Qwen2.5-3B-Instruct (SFT) + DAPO | 243 | Not recorded | 69705d9ae25ad2fa9d1ed223 | | Qwen2.5-3B-Instruct (SFT) + M-GRPO | 223 | Not recorded | 69705d9ae25ad2fa9d1ed223 | | GPT-4o-mini | 123 | Not recorded | 69705d9ae25ad2fa9d1ed223 | | DeepSeek-R1 | 117 | Not recorded | 69705d9ae25ad2fa9d1ed223 | | Qwen2.5-VL-72B-Instruct | 96 | Not recorded | 69705d9ae25ad2fa9d1ed223 | | Qwen2.5-7B-Instruct | 81 | Not recorded | 69705d9ae25ad2fa9d1ed223 | | Qwen2.5-3B-Instruct | 68 | Not recorded | 69705d9ae25ad2fa9d1ed223 | ### Qwen2.5-7B-Instruct (DTFT) Text Score: 285. Result ID: 69992eda5698665ec49767f0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RAG Strategy":"NeuSym RAG","Backbone":"Qwen2.5-7B-Instruct","Fine-tuning":"DTFT"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct (DTFT) + DFPO (PaperGuide) Text Score: 281. Result ID: 69992eda5698665ec49767f1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RAG Strategy":"NeuSym RAG","Backbone":"Qwen2.5-7B-Instruct","Fine-tuning":"DTFT","RL Method":"DFPO"}. This record does not assert independent reproduction. ### Qwen2.5-3B-Instruct (DTFT) + DFPO (PaperGuide) Text Score: 266. Result ID: 69992eda5698665ec49767ec. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RAG Strategy":"NeuSym RAG","Backbone":"Qwen2.5-3B-Instruct","Fine-tuning":"DTFT","RL Method":"DFPO"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct (SFT) Text Score: 266. Result ID: 69992eda5698665ec49767ed. [Reporting paper]() Recorded evaluation configuration: {"RAG Strategy":"NeuSym RAG","Backbone":"Qwen2.5-7B-Instruct","Fine-tuning":"SFT"}. This record does not assert independent reproduction. ### Qwen2.5-3B-Instruct (DTFT) Text Score: 253. Result ID: 69992eda5698665ec49767eb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RAG Strategy":"NeuSym RAG","Backbone":"Qwen2.5-3B-Instruct","Fine-tuning":"DTFT"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct (SFT) + DAPO Text Score: 253. Result ID: 69992eda5698665ec49767ef. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RAG Strategy":"NeuSym RAG","Backbone":"Qwen2.5-7B-Instruct","Fine-tuning":"SFT","RL Method":"DAPO"}. This record does not assert independent reproduction. ### Qwen2.5-3B-Instruct (SFT) Text Score: 247. Result ID: 69992eda5698665ec49767e8. [Reporting paper]() Recorded evaluation configuration: {"RAG Strategy":"NeuSym RAG","Backbone":"Qwen2.5-3B-Instruct","Fine-tuning":"SFT"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct (SFT) + M-GRPO Text Score: 247. Result ID: 69992eda5698665ec49767ee. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RAG Strategy":"NeuSym RAG","Backbone":"Qwen2.5-7B-Instruct","Fine-tuning":"SFT","RL Method":"M-GRPO"}. This record does not assert independent reproduction. ### Qwen2.5-3B-Instruct (SFT) + DAPO Text Score: 243. Result ID: 69992eda5698665ec49767ea. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RAG Strategy":"NeuSym RAG","Backbone":"Qwen2.5-3B-Instruct","Fine-tuning":"SFT","RL Method":"DAPO"}. This record does not assert independent reproduction. ### Qwen2.5-3B-Instruct (SFT) + M-GRPO Text Score: 223. Result ID: 69992eda5698665ec49767e9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RAG Strategy":"NeuSym RAG","Backbone":"Qwen2.5-3B-Instruct","Fine-tuning":"SFT","RL Method":"M-GRPO"}. This record does not assert independent reproduction. ### GPT-4o-mini Text Score: 123. Result ID: 69992eda5698665ec49767e6. [Reporting paper]() Recorded evaluation configuration: {"RAG Strategy":"Classic RAG","Training Strategy":"Prompting"}. This record does not assert independent reproduction. ### DeepSeek-R1 Text Score: 117. Result ID: 69992eda5698665ec49767e7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RAG Strategy":"Classic RAG","Training Strategy":"Prompting"}. This record does not assert independent reproduction. ### Qwen2.5-VL-72B-Instruct Text Score: 96. Result ID: 69992eda5698665ec49767e5. [Reporting paper]() Recorded evaluation configuration: {"RAG Strategy":"Classic RAG","Training Strategy":"Prompting"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct Text Score: 81. Result ID: 69992eda5698665ec49767e4. [Reporting paper]() Recorded evaluation configuration: {"RAG Strategy":"Classic RAG","Training Strategy":"Prompting"}. This record does not assert independent reproduction. ### Qwen2.5-3B-Instruct Text Score: 68. Result ID: 69992eda5698665ec49767e3. [Reporting paper]() Recorded evaluation configuration: {"RAG Strategy":"Classic RAG","Training Strategy":"Prompting"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.