Retrieval-Augmented Generation on Historical Newspaper Archives 384 queries k=5
86.72Context RelevanceOurs - LLM(Semantic + BM25s) + reranker
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Ours - LLM(Semantic + BM25s) + rerankerk=5, OCR Sanitization=Gemini 1.5 Flash, Retrieval Strategy=Hybrid, Reranker=Enabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 86.72 | 67.4 | 87.79 | |
| Semantic + BM25s + rerankerk=5, OCR Sanitization=None, Retrieval Strategy=Hybrid, Reranker=Enabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 83.92 | 64.71 | 85.73 | |
| LLM(Semantic + BM25s)k=5, OCR Sanitization=Gemini 1.5 Flash, Retrieval Strategy=Hybrid, Reranker=Disabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 75.2 | 60.69 | 71.5 | |
| Semantic + BM25sk=5, OCR Sanitization=None, Retrieval Strategy=Hybrid, Reranker=Disabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 73.83 | 60.22 | 75.42 | |
| BM25sk=5, OCR Sanitization=None, Retrieval Strategy=Lexical, Reranker=Disabled, Embedding Model=N/A, Generator LLM=GPT-4o-mini2026.07 | 69.27 | 52.22 | 65.14 | |
| LLM(Semantic)k=5, OCR Sanitization=Gemini 1.5 Flash, Retrieval Strategy=Dense, Reranker=Disabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 69.08 | 50.66 | 62.37 | |
| LLM(BM25s)k=5, OCR Sanitization=Gemini 1.5 Flash, Retrieval Strategy=Lexical, Reranker=Disabled, Embedding Model=N/A, Generator LLM=GPT-4o-mini2026.07 | 67.97 | 53.12 | 64.78 | |
| Semantick=5, OCR Sanitization=None, Retrieval Strategy=Dense, Reranker=Disabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 67.32 | 51.69 | 63.1 |