List your company
ResearchFormerly WizWand

Long-context Reasoning on BABILong 16k

29.8AccuracyFull (exact)
6.19212.32118.4524.579Apr 7, 2026
Updated 6mo ago

Evaluation results

MethodLinks
Full (exact)Model=QWEN3-1.7B2026.0429.8
Sink + Tail + Top-KModel=QWEN3-1.7B, Load Budget=5%2026.0428.8
MAGICPIGModel=QWEN3-1.7B, Load Budget=4%2026.0428.4
Sink + Tail + Top-K + ϕModel=QWEN3-1.7B, Load Budget=5%2026.0428.3
MAGICPIGModel=QWEN3-1.7B, Load Budget=5%2026.0428.3
Sink + Tail + Top-KModel=QWEN3-1.7B, Load Budget=4%2026.0428
MAGICPIGModel=QWEN3-1.7B, Load Budget=3%2026.0427.8
Sink + Tail + Top-K + ϕModel=QWEN3-1.7B, Load Budget=4%2026.0427.7
SNAPKVModel=QWEN3-1.7B, Load Budget=5%2026.0427.5
Sink + Tail + Top-KModel=QWEN3-1.7B, Load Budget=3%2026.0427.4
Sink + Tail + Top-K + ϕModel=QWEN3-1.7B, Load Budget=3%2026.0427.3
SNAPKVModel=QWEN3-1.7B, Load Budget=4%2026.0427.1
MAGICPIGModel=QWEN3-1.7B, Load Budget=2%2026.0427
SNAPKVModel=QWEN3-1.7B, Load Budget=3%2026.0426.9
Sink + Tail + Top-KModel=QWEN3-1.7B, Load Budget=2%2026.0426.7
Sink + Tail + Top-K + ϕModel=QWEN3-1.7B, Load Budget=2%2026.0426.5
SNAPKVModel=QWEN3-1.7B, Load Budget=2%2026.0426.1
Sink + Tail + Top-KModel=QWEN3-1.7B, Load Budget=1%2026.0425.9
PYRAMIDKVModel=QWEN3-1.7B, Load Budget=5%2026.0425.4
MAGICPIGModel=QWEN3-1.7B, Load Budget=1%2026.0424.9
Sink + Tail + Top-K + ϕModel=QWEN3-1.7B, Load Budget=1%2026.0424.8
SNAPKVModel=QWEN3-1.7B, Load Budget=1%2026.0424.6
Full (exact)Model=LLAMA-3.2-1B-INSTRUCT2026.0424.4
PYRAMIDKVModel=QWEN3-1.7B, Load Budget=4%2026.0423.9
SNAPKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=2%2026.0423.8
SNAPKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=4%2026.0423.8
PYRAMIDKVModel=QWEN3-1.7B, Load Budget=3%2026.0423.8
SNAPKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=3%2026.0423.7
Sink + Tail + Top-K + ϕModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=1%2026.0423.6
MAGICPIGModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=3%2026.0423.6
SNAPKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=5%2026.0423.5
QUESTModel=QWEN3-1.7B, Load Budget=5%2026.0423.5
MAGICPIGModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=4%2026.0423.4
Sink + Tail + Top-K + ϕModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=2%2026.0423
QUESTModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=5%2026.0423
MAGICPIGModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=5%2026.0423
Sink + Tail + Top-K + ϕModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=3%2026.0422.9
Sink + Tail + Top-K + ϕModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=4%2026.0422.9
Sink + Tail + Top-K + ϕModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=5%2026.0422.8
SNAPKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=1%2026.0422.8
MAGICPIGModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=2%2026.0422.7
QUESTModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=4%2026.0422.4
PYRAMIDKVModel=QWEN3-1.7B, Load Budget=2%2026.0422.2
Sink + Tail + Top-KModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=5%2026.0422.1
QUESTModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=3%2026.0422
PYRAMIDKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=4%2026.0422
PYRAMIDKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=5%2026.0421.9
Sink + Tail + Top-KModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=4%2026.0421.8
QUESTModel=QWEN3-1.7B, Load Budget=4%2026.0421.7
PYRAMIDKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=3%2026.0421.6
Sink + Tail + Top-KModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=3%2026.0421.3
MAGICPIGModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=1%2026.0421.3
PYRAMIDKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=2%2026.0421.3
PYRAMIDKVModel=QWEN3-1.7B, Load Budget=1%2026.0421.1
Sink + Tail + Top-KModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=2%2026.0420.6
Sink + Tail + Top-KModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=1%2026.0420.4
QUESTModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=2%2026.0420.2
QUESTModel=QWEN3-1.7B, Load Budget=3%2026.0420.2
PYRAMIDKVModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=1%2026.0419.5
QUESTModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=1%2026.0417.7
QUESTModel=QWEN3-1.7B, Load Budget=2%2026.0417.4
QUESTModel=QWEN3-1.7B, Load Budget=1%2026.0414.1
STREAMINGLLMModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=5%2026.0413.7
STREAMINGLLMModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=4%2026.0412.9
STREAMINGLLMModel=QWEN3-1.7B, Load Budget=5%2026.0412.5
STREAMINGLLMModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=3%2026.0412.4
STREAMINGLLMModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=2%2026.0411.4
STREAMINGLLMModel=QWEN3-1.7B, Load Budget=4%2026.0411.2
STREAMINGLLMModel=LLAMA-3.2-1B-INSTRUCT, Load Budget=1%2026.0411
STREAMINGLLMModel=QWEN3-1.7B, Load Budget=3%2026.0410.2
STREAMINGLLMModel=QWEN3-1.7B, Load Budget=2%2026.048.7
STREAMINGLLMModel=QWEN3-1.7B, Load Budget=1%2026.047.1