Long-context Information Retrieval on Needle-In-a-Haystack Verbatim prompt (test) benchmark leaderboard · SOTA2 Research

Long-context Information Retrieval on Needle-In-a-Haystack Verbatim prompt (test)

0.996Accuracy (Depth 0%)Vanilla

Evaluation Results

Method	Links
Vanilla2024.06		0.996	1	1	0.998	1	1	0.998	1	1	1	0.998	0.9991
Block Transformer2024.06		0.962	0.976	0.962	0.966	0.984	0.98	0.972	0.988	0.99	0.994	0.962	0.976
Vanilla2024.06		0.956	0.994	0.99	0.994	0.992	0.992	0.99	0.996	0.996	0.99	0.956	0.986
Block Transformer2024.06		0.952	0.994	0.988	0.988	0.988	0.988	0.99	0.994	0.992	0.974	0.996	0.9858
Block Transformer2024.06		0.926	0.984	0.994	0.988	0.996	0.996	0.988	0.998	0.998	0.992	0.98	0.9855
Block Transformer2024.06		0.902	0.994	0.996	0.992	0.986	0.996	0.996	0.998	0.998	0.992	0.992	0.9856
Vanilla2024.06		0.082	0.014	0.03	0.068	0.078	0.126	0.454	0.658	0.634	0.846	0.994	0.3622