Closed-set Reasoning on ARC Challenge (test)
87.3AccuracyMASS-RAG-Qwen3 8B
Evaluation Results
| Method | Links | |
|---|---|---|
| MASS-RAG-Qwen3 8BRetrieval Strategy=Proposed2026.04 | 87.3 | |
| Qwen3 8BRetrieval Strategy=Training-free retrieval2026.04 | 86.4 | |
| MASS-RAG-Llama3 8BRetrieval Strategy=Proposed2026.04 | 78.7 | |
| MASS-RAG-Mistral 7BRetrieval Strategy=Proposed2026.04 | 72.4 | |
| MASS-RAG-Llama2 7BRetrieval Strategy=Proposed2026.04 | 72.2 | |
| Self-RAG 7BRetrieval Strategy=Training-based retrieval2026.04 | 67.3 | |
| Llama2-FT 7BRetrieval Strategy=Training-based retrieval2026.04 | 65.8 | |
| MAIN-RAG-Llama3 8BRetrieval Strategy=Training-free retrieval2026.04 | 61.9 | |
| MAIN-RAG-Mistral 7BRetrieval Strategy=Training-free retrieval2026.04 | 58.9 | |
| Llama3 8BRetrieval Strategy=Without retrieval2026.04 | 58.8 | |
| Mistral 7BRetrieval Strategy=Training-free retrieval2026.04 | 57.1 | |
| Llama3 8BRetrieval Strategy=Training-free retrieval2026.04 | 55.6 | |
| Mistral 7BRetrieval Strategy=Without retrieval2026.04 | 55.5 | |
| Llama2 7BRetrieval Strategy=Training-free retrieval2026.04 | 51 | |
| Llama2 13BRetrieval Strategy=Without retrieval2026.04 | 29.4 | |
| Llama2 13BRetrieval Strategy=Training-free retrieval2026.04 | 26 | |
| Llama2 7BRetrieval Strategy=Without retrieval2026.04 | 21.8 |