Retrieval-Augmented Generation on Historical Newspaper Archives 384 queries k=25
87.17Context RelevanceOurs - LLM(Semantic + BM25s) + reranker
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Ours - LLM(Semantic + BM25s) + rerankerk=25, OCR Sanitization=Gemini 1.5 Flash, Retrieval Strategy=Hybrid, Reranker=Enabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 87.17 | 65.96 | 84.79 | |
| Semantic + BM25s + rerankerk=25, OCR Sanitization=None, Retrieval Strategy=Hybrid, Reranker=Enabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 84.51 | 63.91 | 82.02 | |
| LLM(Semantic + BM25s)k=25, OCR Sanitization=Gemini 1.5 Flash, Retrieval Strategy=Hybrid, Reranker=Disabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 81.07 | 63.81 | 66.93 | |
| Semantic + BM25sk=25, OCR Sanitization=None, Retrieval Strategy=Hybrid, Reranker=Disabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 77.35 | 64.78 | 72.91 | |
| LLM(Semantic)k=25, OCR Sanitization=Gemini 1.5 Flash, Retrieval Strategy=Dense, Reranker=Disabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 75.85 | 58.9 | 62.06 | |
| Semantick=25, OCR Sanitization=None, Retrieval Strategy=Dense, Reranker=Disabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 72.14 | 58.33 | 61.91 | |
| BM25sk=25, OCR Sanitization=None, Retrieval Strategy=Lexical, Reranker=Disabled, Embedding Model=N/A, Generator LLM=GPT-4o-mini2026.07 | 69.99 | 59.25 | 65.77 | |
| LLM(BM25s)k=25, OCR Sanitization=Gemini 1.5 Flash, Retrieval Strategy=Lexical, Reranker=Disabled, Embedding Model=N/A, Generator LLM=GPT-4o-mini2026.07 | 67.58 | 59.42 | 65.36 |