Retrieval-Augmented Generation on Historical Newspaper Archives 384 queries k=10
87.63Context RelevanceOurs - LLM(Semantic + BM25s) + reranker
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Ours - LLM(Semantic + BM25s) + rerankerk=10, OCR Sanitization=Gemini 1.5 Flash, Retrieval Strategy=Hybrid, Reranker=Enabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 87.63 | 65.92 | 87.05 | |
| Semantic + BM25s + rerankerk=10, OCR Sanitization=None, Retrieval Strategy=Hybrid, Reranker=Enabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 85.03 | 63.6 | 84.94 | |
| LLM(Semantic + BM25s)k=10, OCR Sanitization=Gemini 1.5 Flash, Retrieval Strategy=Hybrid, Reranker=Disabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 77.99 | 62.62 | 69.71 | |
| Semantic + BM25sk=10, OCR Sanitization=None, Retrieval Strategy=Hybrid, Reranker=Disabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 75.39 | 61.43 | 74.35 | |
| LLM(Semantic)k=10, OCR Sanitization=Gemini 1.5 Flash, Retrieval Strategy=Dense, Reranker=Disabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 73.18 | 54.74 | 62.93 | |
| Semantick=10, OCR Sanitization=None, Retrieval Strategy=Dense, Reranker=Disabled, Embedding Model=text-embedding-2-preview, Generator LLM=GPT-4o-mini2026.07 | 71.29 | 54.73 | 63.69 | |
| BM25sk=10, OCR Sanitization=None, Retrieval Strategy=Lexical, Reranker=Disabled, Embedding Model=N/A, Generator LLM=GPT-4o-mini2026.07 | 70.51 | 56.15 | 65.99 | |
| LLM(BM25s)k=10, OCR Sanitization=Gemini 1.5 Flash, Retrieval Strategy=Lexical, Reranker=Disabled, Embedding Model=N/A, Generator LLM=GPT-4o-mini2026.07 | 69.6 | 57.45 | 65.51 |