List your company
ResearchFormerly WizWand

Question Answering on NQ (EM, F1, Token Count)

41.88Exact Match (EM)CORE (1.5B)
20.539226.079631.6237.1604Aug 24, 2025
Updated 5mo ago

Evaluation results

MethodLinks
CORE (1.5B)Compression setting=Generalization to top 10 documents, Training setting=Trained on top 5 documents, Method=CORE (1.5B)2025.0841.8851.2652
CORE (1.5B)Compression setting=Compression of top 5 documents, Method=CORE (1.5B)2025.0841.0250.446
Top 10 DocumentsCompression=None, Retrieval=Top 10 Documents2025.0838.6750.031,425
Top 5 DocumentsCompression=None, Retrieval=Top 5 Documents2025.0838.0349.16712
Top 3 DocumentsCompression=None, Retrieval=Top 3 Documents2025.0837.7848.45427
DeepSeek-V3 (671B)Compression setting=Compression of top 5 documents, Method=DeepSeek-V3 (671B)2025.0837.7350.3954
QGC (1.5B)Compression setting=Compression of top 5 documents, Method=QGC (1.5B)2025.0836.2345.8849
NoiseFilter-IB (1.5B)Compression setting=Compression of top 5 documents, Method=NoiseFilter-IB (1.5B)2025.0835.1545.9448
Top 1 DocumentCompression=None, Retrieval=Top 1 Document2025.0834.4644.41142
RECOMP-Abs (1.5B)Compression setting=Compression of top 5 documents, Method=RECOMP-Abs (1.5B)2025.0834.1846.2658
RECOMP-Ext (1.5B)Compression setting=Compression of top 5 documents, Method=RECOMP-Ext (1.5B)2025.0833.8446.0556
LongLLMLingua (1.5B)Compression setting=Compression of top 5 documents, Method=LongLLMLingua (1.5B)2025.0833.6543.15152
Qwen2.5-1.5B-InstructCompression setting=Compression of top 5 documents, Method=Qwen2.5-1.5B-Instruct2025.0831.9443.0336
BM25Compression setting=Compression of top 5 documents, Method=BM252025.0825.2336.4737
No RetrievalRetrieval=None2025.0821.3630.970