List your company
ResearchFormerly WizWand

Variable Tracking on RULER-VT

99.9AccuracyFP8
32.61250.08167.5585.019Nov 3, 2025
Updated 7mo ago

Evaluation results

MethodLinks
FP8Backbone=Llama 3.1 8B, Compression Ratio (CR)=2, Evaluation Protocol=1-shot2025.1199.9
VanillaBackbone=Llama 3.1 8B, Compression Ratio (CR)=1, Evaluation Protocol=1-shot2025.1199.8
GEAR2-bitBackbone=Llama 3.1 8B, Compression Ratio (CR)=5, Evaluation Protocol=1-shot2025.1199.8
xKVBackbone=Llama 3.1 8B, Compression Ratio (CR)=1-5, Evaluation Protocol=1-shot2025.1199.8
VanillaBackbone=Mistral NeMo 12B, Compression Ratio (CR)=1, Evaluation Protocol=1-shot2025.1199.8
FP8Backbone=Mistral NeMo 12B, Compression Ratio (CR)=2, Evaluation Protocol=1-shot2025.1199.8
TOVABackbone=Llama 3.1 8B, Compression Ratio (CR)=8, Evaluation Protocol=1-shot2025.1199.7
TOVABackbone=Mistral NeMo 12B, Compression Ratio (CR)=8, Evaluation Protocol=1-shot2025.1199.6
kvtc8×Backbone=Mistral NeMo 12B, Compression Ratio (CR)=10-11, Evaluation Protocol=1-shot2025.1199.5
kvtc16×Backbone=Mistral NeMo 12B, Compression Ratio (CR)=17-20, Evaluation Protocol=1-shot2025.1199.5
VanillaBackbone=MN-Minitron 8B, Compression Ratio (CR)=1, Evaluation Protocol=1-shot2025.1199.4
GEAR2-bitBackbone=Mistral NeMo 12B, Compression Ratio (CR)=5, Evaluation Protocol=1-shot2025.1199.4
xKVBackbone=Mistral NeMo 12B, Compression Ratio (CR)=1-5, Evaluation Protocol=1-shot2025.1199.4
TOVABackbone=MN-Minitron 8B, Compression Ratio (CR)=8, Evaluation Protocol=1-shot2025.1199.3
FP8Backbone=MN-Minitron 8B, Compression Ratio (CR)=2, Evaluation Protocol=1-shot2025.1199.2
kvtc8×Backbone=Llama 3.1 8B, Compression Ratio (CR)=9-10, Evaluation Protocol=1-shot2025.1199.1
kvtc16×Backbone=Llama 3.1 8B, Compression Ratio (CR)=18-22, Evaluation Protocol=1-shot2025.1199.1
xKVBackbone=MN-Minitron 8B, Compression Ratio (CR)=1-5, Evaluation Protocol=1-shot2025.1199.1
KIVI2-bitBackbone=Llama 3.1 8B, Compression Ratio (CR)=5, Evaluation Protocol=1-shot2025.1198.9
kvtc32×Backbone=Llama 3.1 8B, Compression Ratio (CR)=34-44, Evaluation Protocol=1-shot2025.1198.9
kvtc32×Backbone=Mistral NeMo 12B, Compression Ratio (CR)=31-43, Evaluation Protocol=1-shot2025.1198.9
kvtc8×Backbone=MN-Minitron 8B, Compression Ratio (CR)=10-11, Evaluation Protocol=1-shot2025.1198.8
kvtc16×Backbone=MN-Minitron 8B, Compression Ratio (CR)=17-21, Evaluation Protocol=1-shot2025.1198.8
GEAR2-bitBackbone=MN-Minitron 8B, Compression Ratio (CR)=5, Evaluation Protocol=1-shot2025.1198.3
KIVI2-bitBackbone=Mistral NeMo 12B, Compression Ratio (CR)=5, Evaluation Protocol=1-shot2025.1198.3
kvtc64×Backbone=Mistral NeMo 12B, Compression Ratio (CR)=51-87, Evaluation Protocol=1-shot2025.1198
KIVI2-bitBackbone=MN-Minitron 8B, Compression Ratio (CR)=5, Evaluation Protocol=1-shot2025.1196.8
kvtc32×Backbone=MN-Minitron 8B, Compression Ratio (CR)=32-46, Evaluation Protocol=1-shot2025.1196
kvtc64×Backbone=Llama 3.1 8B, Compression Ratio (CR)=60-88, Evaluation Protocol=1-shot2025.1195.9
kvtc64×Backbone=MN-Minitron 8B, Compression Ratio (CR)=53-95, Evaluation Protocol=1-shot2025.1193.4
H2OBackbone=Llama 3.1 8B, Compression Ratio (CR)=8, Evaluation Protocol=1-shot2025.1150.4
H2OBackbone=MN-Minitron 8B, Compression Ratio (CR)=8, Evaluation Protocol=1-shot2025.1139.2
H2OBackbone=Mistral NeMo 12B, Compression Ratio (CR)=8, Evaluation Protocol=1-shot2025.1135.2