List your company
ResearchFormerly WizWand

Reasoning on BBH (Accuracy)

74.99BBH AccuracyNTele-R1-32B
24.019637.252350.48563.7177Jul 6, 2025Sep 4, 2025Nov 3, 2025Jan 3, 2026Mar 4, 2026May 3, 2026Jul 3, 2026
Updated 3mo ago

Evaluation results

MethodLinks
NTele-R1-32B2025.0874.99
DS-32B2025.0872.74
OuroModel Version=1.4B R4, Architecture=LoopLM, # Params=1.4B, # Tokens=7.7T, recurrent steps=42025.1071.02
Qwen3Model Version=4B, Architecture=Dense, # Params=4.0B, # Tokens=36T2025.1070.95
Gemma3Model Version=4B, Architecture=Dense, # Params=4.0B, # Tokens=4T2025.1066.32
CODABackbone=Qwen-2.5-7B, Data Budget=50%2026.0765
GradNormBackbone=Qwen-2.5-7B, Data Budget=50%2026.0763.8
PCGradBackbone=Qwen-2.5-7B, Data Budget=50%2026.0763.6
GREATSBackbone=Qwen-2.5-7B, Data Budget=50%2026.0763.4
DoReMiBackbone=Qwen-2.5-7B, Data Budget=50%2026.0763.1
Full dataBackbone=Qwen-2.5-7B, Data Budget=100%2026.0762.8
TemperatureBackbone=Qwen-2.5-7B, Data Budget=50%2026.0762.7
ProportionalBackbone=Qwen-2.5-7B, Data Budget=50%2026.0762.3
Uniform mixingBackbone=Qwen-2.5-7B, Data Budget=50%2026.0762
TTT+CT-KVTraining time per task (s)=172025.0758.2
CT-KVTraining time per task (s)=72025.0757.9
TTTTraining time per task (s)=142025.0757.8
CT-PromptTraining time per task (s)=142025.0756.3
CODABackbone=LLaMA-3.1-8B, Data Budget=50%2026.0756.3
Qwen2.5Model Version=3B, Architecture=Dense, # Params=3.0B, # Tokens=18T2025.1055.37
GradNormBackbone=LLaMA-3.1-8B, Data Budget=50%2026.0755.1
PCGradBackbone=LLaMA-3.1-8B, Data Budget=50%2026.0754.9
GREATSBackbone=LLaMA-3.1-8B, Data Budget=50%2026.0754.7
DoReMiBackbone=LLaMA-3.1-8B, Data Budget=50%2026.0754.4
Full dataBackbone=LLaMA-3.1-8B, Data Budget=100%2026.0754.2
TemperatureBackbone=LLaMA-3.1-8B, Data Budget=50%2026.0754
Qwen3Model Version=1.7B, Architecture=Dense, # Params=1.7B, # Tokens=36T2025.1053.51
ProportionalBackbone=LLaMA-3.1-8B, Data Budget=50%2026.0753.5
Uniform mixingBackbone=LLaMA-3.1-8B, Data Budget=50%2026.0753.2
Prefix TuningTraining time per task (s)=9, m=# demo2025.0752.8
Prefix TuningTraining time per task (s)=7, m=322025.0752.7
DoRATraining time per task (s)=92025.0752.6
LoRATraining time per task (s)=92025.0751.7
Prompt TuningTraining time per task (s)=7, m=322025.0750.8
ICLTraining time per task (s)=0, Evaluation protocol=In-Context Learning2025.0750.4
Prompt TuningTraining time per task (s)=16, m=# demo2025.0747.5
Rank-Stabilized LoRATraining time per task (s)=82025.0746.7
Qwen2.5Model Version=1.5B, Architecture=Dense, # Params=1.5B, # Tokens=18T2025.1043.66
Zero-ShotTraining time per task (s)=0, Evaluation protocol=Zero-Shot2025.0740.9
MERGEvolveSetting=single-task, Number of runs=52026.0639.53
Llama3.2Model Version=3B, Architecture=Dense, # Params=3.0B, # Tokens=9T2025.1039.45
EMMSetting=single-task, Number of runs=52026.0639.35
LoraHubSetting=single-task, Number of runs=52026.0639.3
Pack of LLMsSetting=single-task, Number of runs=52026.0639.2
Model SwarmsSetting=single-task, Number of runs=52026.0638.58
TIESSetting=single-task, Number of runs=52026.0636.2
Expert FusionSetting=single-task, Number of runs=52026.0631.82
Llama3.2Model Version=1.2B, Architecture=Dense, # Params=1.0B, # Tokens=9T2025.1030.72
Gemma3Model Version=1B, Architecture=Dense, # Params=1.0B, # Tokens=2T2025.1030.26
Best Single ExpertSetting=single-task, Number of runs=52026.0630.22
Data MergeSetting=single-task, Number of runs=52026.0625.98