RAG Question Answering on NQ (Natural Questions) (F1)
54.06F1 ScoreIndexLM-1.7B
Evaluation Results
| Method | Links | |
|---|---|---|
| IndexLM-1.7Banswer model=Gemma3-27B-it, Avg. Tokens=20432025.12 | 54.06 | |
| IndexLM-4Banswer model=Gemma3-27B-it, Avg. Tokens=19282025.12 | 53.1 | |
| IndexLM-4Banswer model=Qwen3-4B, Avg. Tokens=19282025.12 | 52.77 | |
| IndexLM-0.6Banswer model=Gemma3-27B-it, Avg. Tokens=19662025.12 | 52.59 | |
| IndexLM-0.6Banswer model=Qwen3-4B, Avg. Tokens=19662025.12 | 51.51 | |
| IndexLM-1.7Banswer model=Qwen3-4B, Avg. Tokens=20432025.12 | 51.42 | |
| Chunk-Rerankanswer model=Gemma3-27B-it, Avg. Tokens=40942025.12 | 49.65 | |
| Qwen3-4B + promptanswer model=Gemma3-27B-it, Avg. Tokens=25632025.12 | 48.02 | |
| Firecrawl Extractanswer model=Gemma3-27B-it, Avg. Tokens=13192025.12 | 47.95 | |
| Firecrawl Extractanswer model=Qwen3-4B, Avg. Tokens=13192025.12 | 47.94 | |
| HtmlRAGanswer model=Gemma3-27B-it, Avg. Tokens=35622025.12 | 46.99 | |
| Qwen3-4B + promptanswer model=Qwen3-4B, Avg. Tokens=25632025.12 | 45.23 | |
| Chunk-Rerankanswer model=Qwen3-4B, Avg. Tokens=40942025.12 | 44.39 | |
| ReaderLM-v2answer model=Qwen3-4B, Avg. Tokens=40002025.12 | 44.35 | |
| ReaderLM-v2answer model=Gemma3-27B-it, Avg. Tokens=40002025.12 | 43.81 | |
| HtmlRAGanswer model=Qwen3-4B, Avg. Tokens=35622025.12 | 42.56 | |
| Markdown (raw)answer model=Gemma3-27B-it, Avg. Tokens=40962025.12 | 40.42 | |
| Markdown (raw)answer model=Qwen3-4B, Avg. Tokens=40962025.12 | 37.74 | |
| HTML (raw)answer model=Qwen3-4B, Avg. Tokens=40962025.12 | 26.32 | |
| HTML (raw)answer model=Gemma3-27B-it, Avg. Tokens=40962025.12 | 24.91 |