List your company
ResearchFormerly WizWand

Long-context understanding on RULER

96ScoreQwen3.5-35B-A3B
44.31257.73171.1584.569Aug 31, 2024Dec 8, 2024Mar 18, 2025Jun 26, 2025Oct 3, 2025Jan 11, 2026Apr 21, 2026
Updated 6mo ago

Evaluation results

MethodLinks
Qwen3.5-35B-A3B#Token=<1002026.0496
JoyAI-LLM Flash#Token=<1002026.0495.7
Qwen2.5-7BSoftmax Ratio=100%2025.1294.45
Gemini-1.5-ProLength=128k2024.0894.4
Qwen3-Next-80B-A3B#Token=1002026.0494.2
Qwen3-30B-A3B#Token=<1002026.0493.7
GA-S2Teacher Model=Qwen2.5-7B, Softmax Ratio=33%2025.1291.1
SMARTTeacher Model=Qwen2.5-7B, Softmax Ratio=33%2025.1289.49
Qwen2.5-1.5BSoftmax Ratio=100%2025.1287.42
GA-S2Teacher Model=Qwen2.5-7B, Softmax Ratio=25%2025.1285.84
SMARTTeacher Model=Qwen2.5-7B, Softmax Ratio=25%2025.1281.58
GPT-4Length=128k2024.0881.2
GLM4-9B-Chat-1MLength=128k2024.0879.9
all-attentionSpeedup @32k=1.0x, Speedup @16k=1.0x2026.0479.4
S1: Distil. Idealized|All–6Speedup @32k=6.13x, Speedup @16k=2.5x2026.0478.7
Gradient-Llama3-8BLength=128k2024.0878.4
Llama3.1-8B-InstructLength=128k2024.0877.7
Yi-34B-200kLength=128k2024.0877.3
LongRecipeModel=Llama3-8B-I, Length=128k2024.0876
FLTModel=Llama3-8B-I, Length=128k2024.0875.8
FLTModel=Llama3-8B-I, Length=80k2024.0875.7
POSEModel=Llama3-8B-I, Length=128k2024.0875.3
GLM-4.7-Flash-T#Token=83002026.0474.7
LongRecipeModel=Llama3-8B-I, Length=80k2024.0874.5
Reg|Lklhd–26Speedup @32k=2.85x, Speedup @16k=1.5x2026.0474.4
Gradient-Llama3-70BLength=128k2024.0872.1
Falcon-H1R 7BSpeedup @32k=4.61x, Speedup @16k=3.4x2026.0472
RPESModel=Llama3-8B-I, Length=128k2024.0871.5
RPESModel=Llama3-8B-I, Length=80k2024.0871.4
Apriel-H1 15BSpeedup @32k=1.97x, Speedup @16k=1.9x2026.0471
LongRecipeModel=Qwen2-7B, Length=80k2024.0870.8
POSEModel=Llama3-8B-I, Length=80k2024.0869.9
Llama3.1-8BLength=128k2024.0869.8
GA-S2Teacher Model=Qwen2.5-1.5B, Softmax Ratio=33%2025.1269.53
FLT*Model=Qwen2-7B, Length=80k2024.0869.5
Apriel-1.6Speedup @32k=1.0x, Speedup @16k=1.0x2026.0469.1
RPESModel=Qwen2-7B, Length=80k2024.0868.9
LongRecipeModel=Mistral-7B, Length=80k2024.0867.2
Idealized|All–18Speedup @32k=1.99x, Speedup @16k=1.1x2026.0467.1
POSEModel=Qwen2-7B, Length=80k2024.0866.7
Llama3.1-70B-InstructLength=128k2024.0866.6
Idealized|Lklhd–6Speedup @32k=6.2x, Speedup @16k=2.4x2026.0466.1
OLMo-Hybrid-Think 7BSpeedup @32k=2.51x, Speedup @16k=2.1x2026.0465.8
Nemotron-Nano 12B v2Speedup @32k=5.85x, Speedup @16k=4.3x2026.0465.2
RPESModel=Mistral-7B, Length=80k2024.0865.1
POSEModel=Mistral-7B, Length=80k2024.0865
Nemotron-3-Nano 30BSpeedup @32k=4.09x, Speedup @16k=2.8x2026.0464.9
LongRecipeModel=Qwen2-7B, Length=128k2024.0864.8
SMARTTeacher Model=Qwen2.5-1.5B, Softmax Ratio=33%2025.1264.79
RPESModel=Qwen2-7B, Length=128k2024.0864.6
Yi-9B-200kLength=128k2024.0862.3
Idealized|All–6Speedup @32k=6.13x, Speedup @16k=2.5x2026.0461.9
Reg|Lklhd–18Speedup @32k=4.76x, Speedup @16k=2.2x2026.0460.5
POSEModel=Qwen2-7B, Length=128k2024.0860.1
LongRecipeModel=Mistral-7B, Length=128k2024.0858.2
FLTModel=Mistral-7B, Length=80k2024.0857.4
Reg|Lklhd–13Speedup @32k=6.9x, Speedup @16k=2.7x2026.0457
Qwen-3.5 27BSpeedup @32k=0.55x, Speedup @16k=0.5x2026.0454.4
GA-S2Teacher Model=Qwen2.5-1.5B, Softmax Ratio=25%2025.1254.08
Qwen2-72B-InstructLength=128k2024.0853.7
RPESModel=Mistral-7B, Length=128k2024.0852.5
Qwen2-7B-InstructLength=128k2024.0852.5
FLT*Model=Qwen2-7B, Length=128k2024.0851.3
SMARTTeacher Model=Qwen2.5-1.5B, Softmax Ratio=25%2025.1250.98
Reg|Lklhd–10Speedup @32k=10.69x, Speedup @16k=4.2x2026.0448.6
POSEModel=Mistral-7B, Length=128k2024.0846.3