List your company
ResearchFormerly WizWand

Long-context Reasoning on BABILong 8k

34.7AccuracyFull (exact)
7.24414.37221.528.628Apr 7, 2026
Updated 6mo ago

Evaluation results

MethodLinks
Full (exact)Model=QWEN3-1.7B2026.0434.7
Sink + Tail + Top-K + ϕModel=QWEN3-1.7B, Load Budget=5%2026.0433
MAGICPIGModel=QWEN3-1.7B, Load Budget=5%2026.0433
Sink + Tail + Top-K + ϕModel=QWEN3-1.7B, Load Budget=4%2026.0432.5
MAGICPIGModel=QWEN3-1.7B, Load Budget=3%2026.0432.5
Sink + Tail + Top-K + ϕModel=QWEN3-1.7B, Load Budget=3%2026.0432.3
Sink + Tail + Top-KModel=QWEN3-1.7B, Load Budget=5%2026.0432.2
Sink + Tail + Top-KModel=QWEN3-1.7B, Load Budget=4%2026.0432.1
MAGICPIGModel=QWEN3-1.7B, Load Budget=4%2026.0431.9
Sink + Tail + Top-KModel=QWEN3-1.7B, Load Budget=3%2026.0431.6
Sink + Tail + Top-KModel=QWEN3-1.7B, Load Budget=2%2026.0431.3
Sink + Tail + Top-K + ϕModel=QWEN3-1.7B, Load Budget=2%2026.0431.3
SNAPKVModel=QWEN3-1.7B, Load Budget=5%2026.0430.6
SNAPKVModel=QWEN3-1.7B, Load Budget=3%2026.0430.3
MAGICPIGModel=QWEN3-1.7B, Load Budget=2%2026.0430.1
SNAPKVModel=QWEN3-1.7B, Load Budget=4%2026.0429.9
SNAPKVModel=QWEN3-1.7B, Load Budget=2%2026.0429.5
Full (exact)Model=LLAMA-3.2-1B-INSTRUCT2026.0428.9
PYRAMIDKVModel=QWEN3-1.7B, Load Budget=5%2026.0428.9
SNAPKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=5%2026.0428.8
MAGICPIGModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=3%2026.0428.5
SNAPKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=4%2026.0428.4
SNAPKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=3%2026.0428.1
MAGICPIGModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=5%2026.0427.9
PYRAMIDKVModel=QWEN3-1.7B, Load Budget=4%2026.0427.9
Sink + Tail + Top-K + ϕModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=1%2026.0427.7
Sink + Tail + Top-K + ϕModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=5%2026.0427.5
Sink + Tail + Top-K + ϕModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=4%2026.0427.3
MAGICPIGModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=4%2026.0427.3
Sink + Tail + Top-K + ϕModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=3%2026.0426.9
Sink + Tail + Top-K + ϕModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=2%2026.0426.8
SNAPKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=2%2026.0426.8
Sink + Tail + Top-KModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=5%2026.0426.7
MAGICPIGModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=2%2026.0426.7
Sink + Tail + Top-KModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=3%2026.0426.5
Sink + Tail + Top-KModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=4%2026.0426.5
PYRAMIDKVModel=QWEN3-1.7B, Load Budget=3%2026.0426.5
QUESTModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=4%2026.0426.4
QUESTModel=QWEN3-1.7B, Load Budget=5%2026.0426.3
Sink + Tail + Top-KModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=2%2026.0425.7
Sink + Tail + Top-KModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=1%2026.0425.4
PYRAMIDKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=5%2026.0425.4
QUESTModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=5%2026.0425.2
SNAPKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=1%2026.0425.2
QUESTModel=QWEN3-1.7B, Load Budget=4%2026.0425.2
PYRAMIDKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=4%2026.0424.6
QUESTModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=3%2026.0424.5
MAGICPIGModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=1%2026.0424.5
PYRAMIDKVModel=QWEN3-1.7B, Load Budget=2%2026.0424.4
PYRAMIDKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=3%2026.0424.3
QUESTModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=2%2026.0422.8
PYRAMIDKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=2%2026.0422.8
QUESTModel=QWEN3-1.7B, Load Budget=3%2026.0422.2
PYRAMIDKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=1%2026.0420.2
QUESTModel=QWEN3-1.7B, Load Budget=2%2026.0419.4
QUESTModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=1%2026.0418.7
STREAMINGLLMModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=5%2026.0415
STREAMINGLLMModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=4%2026.0414.3
STREAMINGLLMModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=3%2026.0413.2
STREAMINGLLMModel=QWEN3-1.7B, Load Budget=5%2026.0412.6
STREAMINGLLMModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=2%2026.0411.9
STREAMINGLLMModel=QWEN3-1.7B, Load Budget=4%2026.0411.4
STREAMINGLLMModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=1%2026.0411.3
STREAMINGLLMModel=QWEN3-1.7B, Load Budget=3%2026.049.8
STREAMINGLLMModel=QWEN3-1.7B, Load Budget=2%2026.048.3