Retrieval-Augmented Generation on NQ (test)
36.2METEOROracle Retriever
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Oracle RetrieverReader Model=mistral-7b-instruct-v0.22024.06 | 36.2 | 74.2 | 23.6 | — | |
| TrioReader Model=mistral-7b-instruct-v0.22024.06 | 34.5 | 72.4 | 21.3 | — | |
| SGPT + MPNETReader Model=mistral-7b-instruct-v0.22024.06 | 34.4 | 72.4 | 21.2 | — | |
| SGPTReader Model=mistral-7b-instruct-v0.22024.06 | 32.5 | 71.9 | 20 | — | |
| SGPT + BM25Reader Model=mistral-7b-instruct-v0.22024.06 | 32.5 | 71.9 | 20.1 | — | |
| MPNET + BM25Reader Model=mistral-7b-instruct-v0.22024.06 | 28.9 | 70.7 | 17.5 | — | |
| MPNETReader Model=mistral-7b-instruct-v0.22024.06 | 28.6 | 70.6 | 17.3 | — | |
| BM25Reader Model=mistral-7b-instruct-v0.22024.06 | 25.1 | 69.7 | 15.5 | — | |
| GRC-1.7B (no doc)Doc KV cache size=02026.05 | — | — | — | 26.2 | |
| GRC-1.7B (w/. compressed doc)Doc KV cache size=O(1)2026.05 | — | — | — | 29 | |
| GRC-1.7B (w/. plain text doc)Doc KV cache size=O(N)2026.05 | — | — | — | 33.6 | |
| GRC-4B (no doc)Doc KV cache size=02026.05 | — | — | — | 35.4 | |
| GRC-4B (w/. compressed doc)Doc KV cache size=O(1)2026.05 | — | — | — | 40.4 | |
| GRC-4B (w/. plain text doc)Doc KV cache size=O(N)2026.05 | — | — | — | 45.8 | |
| GritLM-7BDoc KV cache size=O(N)2026.05 | — | — | — | 31.6 | |
| GritLM-7B (no doc)Doc KV cache size=02026.05 | — | — | — | 21.4 |