Question Answering on ASQA
50.83StrEMAMATA 8B
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| AMATA 8BModel Scale=8B, Backbone Architecture=AMATA2026.05 | 50.83 | 51.57 | 96.92 | — | — | — | — | — | — | |
| MultiQuery + GeoRAGCandidates=top-200→52026.06 | 49.4 | — | — | 56.4 | — | — | — | — | — | |
| HyDE + GeoRAGCandidates=top-200→52026.06 | 48.1 | — | — | 55.4 | — | — | — | — | — | |
| GraphRAG + GeoRAGCandidates=top-200→52026.06 | 47.6 | — | — | 56.4 | — | — | — | — | — | |
| Hybrid RRF + GeoRAGCandidates=top-200→52026.06 | 47.3 | — | — | 55.7 | — | — | — | — | — | |
| SPA-RL 8BModel Scale=8B, Backbone Architecture=SPA-RL2026.05 | 46.63 | 45.57 | 94.93 | — | — | — | — | — | — | |
| Dense + GeoRAGCandidates=top-200→52026.06 | 46.1 | — | — | 54.7 | — | — | — | — | — | |
| GiGPO 8BModel Scale=8B, Backbone Architecture=GiGPO2026.05 | 45.75 | 46.98 | 95.62 | — | — | — | — | — | — | |
| SMART 8BModel Scale=8B, Backbone Architecture=SMART2026.05 | 45.74 | 44.95 | 94.8 | — | — | — | — | — | — | |
| No RefinementBackbone=Qwen2.5-14B-Instruct2025.02 | 44.73 | — | — | — | — | — | — | — | — | |
| GPT4oBackbone Architecture=GPT-4o2026.05 | 44.02 | 36.92 | 47.53 | — | — | — | — | — | — | |
| BM25 + GeoRAGCandidates=top-200→52026.06 | 43.9 | — | — | 53.2 | — | — | — | — | — | |
| RankCoTBackbone=Qwen2.5-14B-Instruct2025.02 | 41.94 | — | — | — | — | — | — | — | — | |
| Llama3.1-70B+Hit-RAGFramework=Hit-RAG2026.03 | 41.1 | 38.3 | 87.6 | — | — | — | — | — | — | |
| RankCoTBackbone=Llama3-8B-Instruct2025.02 | 41.02 | — | — | — | — | — | — | — | — | |
| Ret-ChatGPTRetrieval=None2026.03 | 40.7 | 39.9 | 79.7 | — | — | — | — | — | — | |
| MultiQuery (direct)Candidates=top-52026.06 | 40.6 | — | — | 49.5 | — | — | — | — | — | |
| Llama3.1-8B+Hit-RAGFramework=Hit-RAG2026.03 | 40.5 | 38.8 | 84.2 | — | — | — | — | — | — | |
| CoTBackbone=Llama3-8B-Instruct2025.02 | 40.25 | — | — | — | — | — | — | — | — | |
| RerankBackbone=Llama3-8B-Instruct2025.02 | 39.92 | — | — | — | — | — | — | — | — | |
| HyDE (direct)Candidates=top-52026.06 | 39.8 | — | — | 48.8 | — | — | — | — | — | |
| MAIN-RAG-Llama38BRetrieval=Training-free, Backbone=Llama3-8B2024.12 | 39.2 | 42 | 70.6 | — | — | — | — | — | — | |
| Hybrid RRF (direct)Candidates=top-52026.06 | 38.9 | — | — | 49.2 | — | — | — | — | — | |
| No RefinementBackbone=Llama3-8B-Instruct2025.02 | 38.79 | — | — | — | — | — | — | — | — | |
| Qwen3-8B+Hit-RAGFramework=Hit-RAG2026.03 | 38.4 | 40.6 | 85.9 | — | — | — | — | — | — | |
| Qwen3-32B+Hit-RAGFramework=Hit-RAG2026.03 | 38.3 | 41.7 | 85 | — | — | — | — | — | — | |
| GraphRAG (direct)Candidates=top-52026.06 | 38.1 | — | — | 49.7 | — | — | — | — | — | |
| SummaryBackbone=Llama3-8B-Instruct2025.02 | 37.81 | — | — | — | — | — | — | — | — | |
| Llama38BRetrieval=Training-free2024.12 | 37.1 | 36.5 | 63 | — | — | — | — | — | — | |
| Alpaca-13BRetrieval=Yes2026.03 | 36.7 | 56.6 | 2 | — | — | — | — | — | — | |
| Dense (direct)Candidates=top-52026.06 | 36.7 | — | — | 47.2 | — | — | — | — | — | |
| RankCoTBackbone=MiniCPM3-4B2025.02 | 35.85 | — | — | — | — | — | — | — | — | |
| MAIN-RAG-Mistral7BRetrieval=Training-free, Backbone=Mistral-7B2024.12 | 35.7 | 36.2 | 60 | — | — | — | — | — | — | |
| Qwen3-4B+Hit-RAGFramework=Hit-RAG2026.03 | 35.5 | 37.9 | 80.1 | — | — | — | — | — | — | |
| ChatGPTRetrieval=None2026.03 | 35.3 | 36.2 | 68.8 | — | — | — | — | — | — | |
| Alpaca13BRetrieval=Training-free2024.12 | 34.8 | 36.7 | 56.6 | — | — | — | — | — | — | |
| BM25 (direct)Candidates=top-52026.06 | 34.2 | — | — | 46 | — | — | — | — | — | |
| Ret-Llama2-chat13BData=Proprietary, Retrieval=Yes2024.12 | 32.8 | 34.8 | 43.8 | — | — | — | — | — | — | |
| Mistral7BRetrieval=Training-free2024.12 | 32.4 | 34.8 | 54.3 | — | — | — | — | — | — | |
| Self-RAG-13BRetrieval=Yes2026.03 | 31.7 | 37 | 71.6 | — | — | — | — | — | — | |
| Llama2-FT7BRetrieval=Training-based2024.12 | 31 | 35.8 | 51.2 | — | — | — | — | — | — | |
| SelfRag 8BModel Scale=8B, Backbone Architecture=Self-RAG2026.05 | 30.95 | 35.98 | 87.12 | — | — | — | — | — | — | |
| Alpaca7BRetrieval=Training-free2024.12 | 30.9 | 33.3 | 57.9 | — | — | — | — | — | — | |
| Self-RAG7BRetrieval=Training-based2024.12 | 30 | 35.7 | 74.3 | — | — | — | — | — | — | |
| Self-RAG-7BRetrieval=Yes2026.03 | 30 | 35.7 | 74.3 | — | — | — | — | — | — | |
| RADIT 8BModel Scale=8B, Backbone Architecture=RADIT2026.05 | 29.01 | 17.22 | 13.86 | — | — | — | — | — | — | |
| Qwen3-32BRetrieval=None2026.03 | 27.4 | 30.3 | 45.8 | — | — | — | — | — | — | |
| No RefinementBackbone=MiniCPM3-4B2025.02 | 24.31 | — | — | — | — | — | — | — | — | |
| Alpaca13BRetrieval=None2024.12 | 22.9 | 32 | 70.6 | — | — | — | — | — | — | |
| Alpaca-13BRetrieval=None2026.03 | 22.9 | 32 | 70.6 | — | — | — | — | — | — | |
| Llama2-chat13BData=Proprietary, Retrieval=No2024.12 | 22.4 | 29.6 | 28.6 | — | — | — | — | — | — | |
| Qwen3-8BRetrieval=None2026.03 | 20.9 | 23.8 | 38.8 | — | — | — | — | — | — | |
| Llama3.1-70BRetrieval=None2026.03 | 20.8 | 20.5 | 40.3 | — | — | — | — | — | — | |
| Llama38BRetrieval=None2024.12 | 19.4 | 30.3 | 54.3 | — | — | — | — | — | — | |
| Llama-3-Ins.8BModel Scale=8B, Backbone Architecture=Llama-3-Ins2026.05 | 18.85 | 33.48 | 37.65 | — | — | — | — | — | — | |
| Alpaca7bRetrieval=None2024.12 | 18.8 | 29.4 | 61.7 | — | — | — | — | — | — | |
| Llama3.1-8BRetrieval=None2026.03 | 18.6 | 20.7 | 35.1 | — | — | — | — | — | — | |
| Qwen3-4BRetrieval=None2026.03 | 17.9 | 22.4 | 25.2 | — | — | — | — | — | — | |
| Llama213BRetrieval=Training-free2024.12 | 16.3 | 20.5 | 24.7 | — | — | — | — | — | — | |
| Llama27BRetrieval=Training-free2024.12 | 16.2 | 23.4 | 33.1 | — | — | — | — | — | — | |
| Mistral7BRetrieval=None2024.12 | 11.2 | 18.1 | 27.6 | — | — | — | — | — | — | |
| Llama27BRetrieval=None2024.12 | 7.9 | 15.3 | 19 | — | — | — | — | — | — | |
| Llama213BRetrieval=None2024.12 | 7.2 | 12.4 | 16 | — | — | — | — | — | — | |
| ActIdxModel=Q-7B2025.06 | — | — | — | — | 27.6 | 30.9 | 31.1 | — | — | |
| ActIdxModel=Q-3B2025.06 | — | — | — | — | 21.4 | 23.9 | 24.2 | — | — | |
| ActIdx-BModel=Q-7B2025.06 | — | — | — | — | 25.4 | 31.4 | 31.9 | — | — | |
| ActIdx-BModel=Q-3B2025.06 | — | — | — | — | 19.7 | 24.5 | 24.9 | — | — | |
| ActIdx-FModel=Q-7B2025.06 | — | — | — | — | 25.8 | 26.7 | 27.9 | — | — | |
| ActIdx-FModel=Q-3B2025.06 | — | — | — | — | 19.8 | 22.6 | 23.1 | — | — | |
| Align-RAGMethod Category=Reflective / Critique Methods2026.05 | — | — | — | — | 34.2 | — | — | 31.8 | 36.1 | |
| bge-m3Generator=Qwen/Qwen2.5-32B-Instruct, Reranking=No, Top-k=52026.02 | — | — | — | 69.4 | — | — | — | — | — | |
| CoTMethod Category=Reasoning and Retrieval Baselines2026.05 | — | — | — | — | 48.5 | — | — | 10.8 | 33.3 | |
| CRITIC-R1Method Category=Reflective / Critique Methods2026.05 | — | — | — | — | 51.6 | — | — | 52 | 66.4 | |
| DeepSeekquery placement=reverse2026.01 | — | — | — | 47.99 | — | — | — | — | — | |
| DeepSeekquery placement=vanilla2026.01 | — | — | — | 45.62 | — | — | — | — | — | |
| few-shotModel=GPT-4.12025.06 | — | — | — | — | 52.7 | 23 | 24 | — | — | |
| GPT-4oquery placement=reverse2026.01 | — | — | — | 49.16 | — | — | — | — | — | |
| GPT-4oquery placement=vanilla2026.01 | — | — | — | 45.13 | — | — | — | — | — | |
| InsOnlyModel=Q-7B2025.06 | — | — | — | — | 19.1 | 20 | 21.2 | — | — | |
| InsOnlyModel=Q-3B2025.06 | — | — | — | — | 15.9 | 3.7 | 4.1 | — | — | |
| InstructRAGRanking Strategy=None2026.04 | — | — | — | — | 66.98 | — | — | — | — | |
| InstructRAGRanking Strategy=Reranker2026.04 | — | — | — | — | 71.51 | — | — | — | — | |
| InstructRAGRanking Strategy=RankCoT2026.04 | — | — | — | — | 67.72 | — | — | — | — | |
| Llama 80Bquery placement=reverse2026.01 | — | — | — | 48.58 | — | — | — | — | — | |
| Llama 80Bquery placement=vanilla2026.01 | — | — | — | 48.33 | — | — | — | — | — | |
| Llama 8Bquery placement=reverse2026.01 | — | — | — | 42.04 | — | — | — | — | — | |
| Llama 8Bquery placement=vanilla2026.01 | — | — | — | 41.81 | — | — | — | — | — | |
| Mistral 7Bquery placement=reverse2026.01 | — | — | — | 46.04 | — | — | — | — | — | |
| Mistral 7Bquery placement=vanilla2026.01 | — | — | — | 43.67 | — | — | — | — | — | |
| Naive-RAGMethod Category=Reasoning and Retrieval Baselines2026.05 | — | — | — | — | 42.8 | — | — | 9.5 | 32.2 | |
| Neuro-RITGenerator=Mistral-7B-Instruct-v0.22026.04 | — | — | — | — | 72.57 | — | — | — | — | |
| Neuro-RITRanking Strategy=None2026.04 | — | — | — | — | 75.31 | — | — | — | — | |
| Neuro-RITRanking Strategy=Reranker2026.04 | — | — | — | — | 80.48 | — | — | — | — | |
| Neuro-RITRanking Strategy=RankCoT2026.04 | — | — | — | — | 69.09 | — | — | — | — | |
| Neuro-RIT + RankCoTGenerator=Mistral-7B-Instruct-v0.22026.04 | — | — | — | — | 69.62 | — | — | — | — | |
| Neuro-RIT + RerankerGenerator=Mistral-7B-Instruct-v0.22026.04 | — | — | — | — | 75.42 | — | — | — | — | |
| No compressionBackbone=Mistral-7B, Compressor=No compression, Inference (Tera-Floating point operations)=20.33, Compression (Tera-Floating point operations)=0., Total (Tera-Floating point operations)=20.332025.03 | — | — | — | — | 75 | — | — | — | — | |
| No compressionBackbone=Llama-1B, Compressor=No compression, Inference (Tera-Floating point operations)=2.85, Compression (Tera-Floating point operations)=0., Total (Tera-Floating point operations)=2.852025.03 | — | — | — | — | 61 | — | — | — | — | |
| No compressionBackbone=Qwen-7B, Compressor=No compression, Inference (Tera-Floating point operations)=18.94, Compression (Tera-Floating point operations)=0., Total (Tera-Floating point operations)=18.942025.03 | — | — | — | — | 70 | — | — | — | — | |
| No compressionBackbone=Mistral-24B, Compressor=No compression, Inference (Tera-Floating point operations)=64.29, Compression (Tera-Floating point operations)=0., Total (Tera-Floating point operations)=64.292025.03 | — | — | — | — | 74 | — | — | — | — |