List your company
ResearchFormerly WizWand

Accuracy on Big-Bench Hard (BBH)

82.3AccuracyMPO+
2.32423.08743.8564.613Jun 16, 2025Aug 17, 2025Oct 19, 2025Dec 20, 2025Feb 21, 2026Apr 24, 2026Jun 26, 2026
Updated 4mo ago

Evaluation results

MethodLinks
MPO+Backbone=olmo-3.1-32b-it-sft2026.0482.3
Pair+Backbone=olmo-3.1-32b-it-sft2026.0482.1
DPOBackbone=olmo-3.1-32b-it-sft2026.0481.6
Mag+Backbone=olmo-3.1-32b-it-sft2026.0481.6
Soft+Backbone=olmo-3.1-32b-it-sft2026.0481.6
RFTBackbone=olmo-3.1-32b-it-sft2026.0477.6
Pair+Backbone=olmo-3-7b-it-sft2026.0473.8
MPO+Backbone=olmo-3-7b-it-sft2026.0473.6
Mag+Backbone=olmo-3-7b-it-sft2026.0473.2
Soft+Backbone=olmo-3-7b-it-sft2026.0472.9
DPOBackbone=olmo-3-7b-it-sft2026.0471.5
RFTBackbone=olmo-3-7b-it-sft2026.0466.7
LoRA# Trainable Parameters=3407.87K, Backbone=Llama-3.1-8B2025.0760.39
(IA)3# Trainable Parameters=524.29K, Backbone=Llama-3.1-8B2025.0760.29
BaseBackbone=olmo-3.1-32b-it-sft2026.0460
SITE (M = 50)# Trainable Parameters=1.02K, Backbone=Llama-3.1-8B, M=502025.0758.04
SITE (M = 1)# Trainable Parameters=1.02K, Backbone=Llama-3.1-8B, M=12025.0756.76
MPO+Backbone=gemma-3-4b-sft2026.0451.7
Soft+Backbone=gemma-3-4b-sft2026.0451.4
I2CL# Trainable Parameters=0.13K, Backbone=Llama-3.1-8B2025.0750.6
DPOBackbone=gemma-3-4b-sft2026.0450.5
Pair+Backbone=gemma-3-4b-sft2026.0450
RFTBackbone=gemma-3-4b-sft2026.0449.9
Mag+Backbone=gemma-3-4b-sft2026.0449.3
GRPOBackbone=Qwen2.5-Math-7B2025.1047.7
BaseBackbone=Qwen2.5-Math-7B2025.1047.5
SePTBackbone=Qwen2.5-Math-7B2025.1047.3
10-shot ICLBackbone=Llama-3.1-8B2025.0747.17
BaseBackbone=olmo-3-7b-it-sft2026.0446
DenseBase Model=Llama-3.2-3B, Model architecture type=Dense, Few-shot CoT=true2025.0644.1
MTV# Trainable Parameters=1.02K, Backbone=Llama-3.1-8B2025.0742.54
MoBBase Model=Llama-3.2-3B, Model architecture type=MoB, Few-shot CoT=true2025.0642.2
TVBackbone=Llama-3.1-8B2025.0742.01
MiCRoBase Model=Llama-3.2-3B, Model architecture type=MiCRo, Few-shot CoT=true2025.0642
BaseBackbone=gemma-3-4b-sft2026.0440.4
Prompt Tuning# Trainable Parameters=81.92K, Backbone=Llama-3.1-8B2025.0733.45
Original ModelBackbone=Llama-3-8B, Avg. Sparsity (L/H)=0% / 0%, Avg. Mem (GB)=4.50, Time (s)=34522026.0630.5
DenseBase Model=Llama-3.2-1B, Model architecture type=Dense, Few-shot CoT=true2025.0630.4
MiCRoBase Model=Llama-3.2-1B, Model architecture type=MiCRo, Few-shot CoT=true2025.0629.8
Learning to AllocateBackbone=Llama-3-8B, Avg. Sparsity (L/H)=34% / 24%, Avg. Mem (GB)=4.10, Time (s)=23802026.0629.8
FlexiDepthBackbone=Llama-3-8B, Avg. Sparsity (L/H)=32% / –, Avg. Mem (GB)=4.05, Time (s)=24202026.0628.2
MoBBase Model=SmollM2-360M, Model architecture type=MoB, Few-shot CoT=true2025.0627.7
Teacher (Qwen3-0.6B)Role=Teacher, Model=Qwen3-0.6B, Evaluation Protocol=generation-based evaluation2026.0327.4
MoBBase Model=Llama-3.2-1B, Model architecture type=MoB, Few-shot CoT=true2025.0627.4
DenseBase Model=SmollM2-360M, Model architecture type=Dense, Few-shot CoT=true2025.0627.3
MiCRoBase Model=SmollM2-360M, Model architecture type=MiCRo, Few-shot CoT=true2025.0627.2
AdaSkipBackbone=Llama-3-8B, Avg. Sparsity (L/H)=35% / –, Avg. Mem (GB)=3.88, Time (s)=23502026.0626.8
IVBackbone=Llama-3.1-8B2025.0726.25
MiCRoBase Model=SmollM2-135M, Model architecture type=MiCRo, Few-shot CoT=true2025.0625.4
Oracle Static PruningBackbone=Llama-3-8B, Avg. Sparsity (L/H)=33% / 20%, Avg. Mem (GB)=3.80, Time (s)=22502026.0624.5
DenseBase Model=SmollM2-135M, Model architecture type=Dense, Few-shot CoT=true2025.0624.1
MoBBase Model=SmollM2-135M, Model architecture type=MoB, Few-shot CoT=true2025.0623.5
Static PruningBackbone=Llama-3-8B, Avg. Sparsity (L/H)=33% / 0%, Avg. Mem (GB)=3.92, Time (s)=23102026.0621.3
Hybrid KDASequence Mixer=KDA, Attention Layers=7, Evaluation Protocol=generation-based evaluation2026.0318.9
FVBackbone=Llama-3.1-8B2025.0717.82
0-shot ICLBackbone=Llama-3.1-8B2025.0716.41
LIVE# Trainable Parameters=131.10K, Backbone=Llama-3.1-8B2025.0712.89
Hybrid MambaSequence Mixer=Mamba2, Attention Layers=7, Evaluation Protocol=generation-based evaluation2026.0312.1
Pure MambaSequence Mixer=Mamba2, Attention Layers=0, Evaluation Protocol=generation-based evaluation2026.038.4
Pure KDASequence Mixer=KDA, Attention Layers=0, Evaluation Protocol=generation-based evaluation2026.035.4