Long context retrieval and reasoning on RULER
94.7ScoreNemotron 3 Ultra
Evaluation Results
| Method | Links | |
|---|---|---|
| Nemotron 3 UltraContext length=1M, Parameter Count=550B-A55B2026.06 | 94.7 | |
| DS-v4-ProContext length=1M, Parameter Count=1.6T-A49B2026.06 | 94.2 | |
| Full KVBackbone=Qwen2.5-14B, KV Cache Budget=Full2026.06 | 90.74 | |
| Qwen-3.5Context length=1M, Parameter Count=397B-17B2026.06 | 90.1 | |
| Full KVBackbone=Llama-3.1-8B, KV Cache Budget=Full2026.06 | 88.35 | |
| DS-v4-FlashContext length=1M, Parameter Count=284B-A13B2026.06 | 87.7 | |
| SEKVBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.06 | 87.34 | |
| SEKVBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.06 | 84.17 | |
| SentenceKVBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.06 | 83.41 | |
| SemantiCacheBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.06 | 82.04 | |
| Full KVBackbone=Llama-3-8B, KV Cache Budget=Full2026.06 | 81.94 | |
| ChunkKVBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.06 | 81.17 | |
| SentenceKVBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.06 | 80.37 | |
| PyramidKVBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.06 | 79.33 | |
| SemantiCacheBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.06 | 79.18 | |
| SEKVBackbone=Llama-3-8B, KV Cache Budget=10%2026.06 | 78.43 | |
| ChunkKVBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.06 | 78.41 | |
| SnapKVBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.06 | 77.48 | |
| PyramidKVBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.06 | 76.23 | |
| SentenceKVBackbone=Llama-3-8B, KV Cache Budget=10%2026.06 | 74.82 | |
| SnapKVBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.06 | 74.31 | |
| Full KVBackbone=Llama-3.2-3B, KV Cache Budget=Full2026.06 | 73.67 | |
| SemantiCacheBackbone=Llama-3-8B, KV Cache Budget=10%2026.06 | 73.47 | |
| ChunkKVBackbone=Llama-3-8B, KV Cache Budget=10%2026.06 | 72.83 | |
| H2OBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.06 | 71.24 | |
| PyramidKVBackbone=Llama-3-8B, KV Cache Budget=10%2026.06 | 70.14 | |
| SnapKVBackbone=Llama-3-8B, KV Cache Budget=10%2026.06 | 68.42 | |
| Full KVBackbone=Mistral-7B, KV Cache Budget=Full2026.06 | 68.38 | |
| H2OBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.06 | 67.82 | |
| SEKVBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.06 | 63.84 | |
| StreamingLLMBackbone=Qwen2.5-14B, KV Cache Budget=10%2026.06 | 61.87 | |
| H2OBackbone=Llama-3-8B, KV Cache Budget=10%2026.06 | 61.33 | |
| SEKVBackbone=Mistral-7B, KV Cache Budget=10%2026.06 | 61.23 | |
| SentenceKVBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.06 | 60.43 | |
| SemantiCacheBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.06 | 59.12 | |
| StreamingLLMBackbone=Llama-3.1-8B, KV Cache Budget=10%2026.06 | 58.43 | |
| ChunkKVBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.06 | 58.34 | |
| SentenceKVBackbone=Mistral-7B, KV Cache Budget=10%2026.06 | 57.84 | |
| SemantiCacheBackbone=Mistral-7B, KV Cache Budget=10%2026.06 | 56.33 | |
| PyramidKVBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.06 | 56.17 | |
| ChunkKVBackbone=Mistral-7B, KV Cache Budget=10%2026.06 | 55.72 | |
| SnapKVBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.06 | 54.83 | |
| PyramidKVBackbone=Mistral-7B, KV Cache Budget=10%2026.06 | 53.41 | |
| StreamingLLMBackbone=Llama-3-8B, KV Cache Budget=10%2026.06 | 52.17 | |
| SnapKVBackbone=Mistral-7B, KV Cache Budget=10%2026.06 | 51.63 | |
| H2OBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.06 | 48.71 | |
| H2OBackbone=Mistral-7B, KV Cache Budget=10%2026.06 | 45.17 | |
| StreamingLLMBackbone=Llama-3.2-3B, KV Cache Budget=10%2026.06 | 41.32 | |
| StreamingLLMBackbone=Mistral-7B, KV Cache Budget=10%2026.06 | 38.24 |