Fact Retrieval on Gov Report
24.4F1 ScoreOpenOrca
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| OpenOrcacontext_window=8k2026.04 | 24.4 | 41 | |
| Nous Hermescontext_window=32k2026.04 | 23.8 | 37 | |
| Full Contextvariant=left2026.04 | 23.4 | 45 | |
| Thought-Retrieverretriever=Contriever, max_context_length=2,000 tokens2026.04 | 23.2 | 50 | |
| Qwen3-Embed-8btype=retriever-based, max_context_length=2,000 tokens2026.04 | 22.9 | 42 | |
| IRCoTtype=retriever-based, max_context_length=2,000 tokens2026.04 | 22.5 | 41 | |
| Contrievertype=retriever-based, max_context_length=2,000 tokens2026.04 | 22.3 | 40 | |
| Full Contextvariant=right2026.04 | 22 | 40 | |
| RECOMPtype=retriever-based, max_context_length=2,000 tokens2026.04 | 21.5 | 35 | |
| BM25type=retriever-based, max_context_length=2,000 tokens2026.04 | 21.1 | 30 | |
| DRAGONtype=retriever-based, max_context_length=2,000 tokens2026.04 | 21 | 40 | |
| TF-IDFtype=retriever-based, max_context_length=2,000 tokens2026.04 | 19.5 | 35 | |
| DPRtype=retriever-based, max_context_length=2,000 tokens2026.04 | 18.8 | 20 |