Retrieval-Augmented Generation on LOFT benchmark leaderboard · SOTA2 Research

Retrieval-Augmented Generation on LOFT

100NQ ScoreMiniLM

Evaluation Results

Method
MiniLM2025.01	100	98	82	—
MiniLM2025.01	100	100	90	—
MiniLM2025.01	100	100	92	—
TinyBERT2025.01	99	98	80	—
TinyBERT2025.01	99	99	81	—
TinyBERT2025.01	99	99	82	—
SFT-RTA w/ RAP2025.01	95	99	100	—
Phi-3-7B-128K2025.01	90	85	63	79
Mistral-2-7B-32K2025.01	89	81	41	70
Oracle RAG2025.01	89	81	41	—
Qwen-2-7B-32K2025.01	88	81	56	75
GPT-4-turbo2025.01	88	87	72	82
GPT-4 Oracle2025.01	88	87	72	—
Mistral-7B Re-ranking (TinyBERT)2025.01	88	78	29	—
Mistral-7B Re-ranking (TinyBERT)2025.01	87	87	33	—
GPT-4-turbo2025.01	85	78	51	71
GPT-4 Vanilla RAG2025.01	85	78	51	—
SFT-RTA w/ RAP2025.01	85	79	39	—
Mistral-7B Re-ranking (MiniLM)2025.01	84	84	39	—
Mistral-7B Re-ranking (MiniLM)2025.01	83	77	27	—
LLaMA-3-instruct-8B2025.01	81	77	60	73
Qwen-2-1.5B-32K2025.01	80	76	44	67
Qwen-2-7B-32K2025.01	79	61	29	56
SFT-DA w/ RAP2025.01	78	76	47	—
Phi-3-7B-128K2025.01	76	68	41	62
SFT-Cite-Context-ID2025.01	76	54	35	—
SFT-Retrieve-then-Answer2025.01	74	69	33	—
SFT-Direct Answer2025.01	70	65	25	—
Mistral-2-7B-32K2025.01	64	62	27	51
Mistral-7B Vanilla RAG2025.01	64	62	27	—
Qwen-2-1.5B-32K2025.01	61	43	12	39
Mistral-7B RAG w/ RTA Prompting2025.01	60	70	27	—
GPT-4-turbo2025.01	58	57	28	48
GPT-4 Close-book2025.01	58	57	28	—
Mistral-2-7B-32K2025.01	46	38	5	30
Mistral-7B Close-book2025.01	46	38	5	—
LLaMA-3-instruct-8B2025.01	45	56	16	39
LLaMA-3-instruct-8B2025.01	41	19	1	20
Phi-3-7B-128K2025.01	39	35	9	28
Qwen-2-7B-32K2025.01	31	30	7	23
Qwen-2-1.5B-32K2025.01	21	22	3	15
RetHead w/ Lgen + Lret2025.01	15	13	7	—