List your company
ResearchFormerly WizWand

Long-context retrieval on RULER (Custom Metrics)

100S1 ScoreFullKV
-4235077Sep 25, 2025
Updated 6mo ago

Evaluation results

MethodLinks
FullKVBase model=LongChat-7b-v1.5-32k, prompt length=16K2025.0910074999174707113
OjaKV-PFBase model=LongChat-7b-v1.5-32k, prompt length=16K, Compression ratio=0.6x2025.099750.791711531.753613.33
OjaKV-PFBase model=LongChat-7b-v1.5-32k, prompt length=16K, Compression ratio=0.8x2025.099554.19927618384713.33
OjaKVBase model=LongChat-7b-v1.5-32k, prompt length=16K, Compression ratio=0.6x2025.099439.995137832.2534.2523.42
OjaKVBase model=LongChat-7b-v1.5-32k, prompt length=16K, Compression ratio=0.8x2025.099044.8550401355.7544.7520.42
StaticPCABase model=LongChat-7b-v1.5-32k, prompt length=16K, Compression ratio=0.8x2025.096828.371723447.7525.7513.08
StaticPCABase model=LongChat-7b-v1.5-32k, prompt length=16K, Compression ratio=0.6x2025.095923.44520330.753115.33
PaluBase model=LongChat-7b-v1.5-32k, prompt length=16K, Compression ratio=0.8x2025.0900000000
PaluBase model=LongChat-7b-v1.5-32k, prompt length=16K, Compression ratio=0.6x2025.0900000000