List your company
ResearchFormerly WizWand

Commonsense Reasoning on SIQA

89.85AccuracyIn-Squeeze
41.85454.314566.77579.2355Sep 20, 2023Mar 1, 2024Aug 11, 2024Jan 21, 2025Jul 3, 2025Dec 13, 2025May 26, 2026
Updated 4mo ago

Evaluation results

MethodLinks
In-Squeezerank=128 to 1, schedule=Min steps2026.0289.85——
In-Squeezerank=128 to 1, schedule=Standard2026.0289.77——
Direct Fine-tuningrank=1, additional_steps=2002026.0289.35——
Cont-Squeezerank=128 to 1, additional_steps=7002026.0289.35——
Cont-Squeezerank=128 to 1, additional_steps=2002026.0289.33——
Direct Fine-tuningrank=1, additional_steps=02026.0288.78——
Cont-Squeezerank=128 to 1, additional_steps=02026.0288.13——
Direct Fine-tuningrank=1, additional_steps=7002026.0287.87——
BaLoRAModel=Llama-3-8B, Params (%)=0.71452026.0481.78——
FFTModel=Llama-3.2-3B, #Params=3.21B2025.0981.49——
BoHAModel=Llama-3.2-3B, #Params=48.63M2025.0981.35——
HiRAModel=Llama-3-8B, Params (%)=0.70022026.0481.15——
GraLoRAModel=Llama-3.2-3B, #Params=48.63M2025.0981.03——
ABBAModel=Llama-3.2-3B, #Params=48.63M2025.0981——
HiRAModel=Llama-3.2-3B, #Params=48.63M2025.0980.81——
MoRAModel=Llama-3-8B, Params (%)=0.69972026.0480.71——
BaLoRAModel=Llama-2-7B, Params (%)=0.84272026.0480.45——
LoRAModel=Llama-3-8B, Params (%)=0.70022026.0479.9——
DoRAModel=Llama-3-8B, Params (%)=0.70022026.0479.9——
LoRAModel=Llama-3.2-3B, #Params=48.63M2025.0979.84——
DoRAModel=Llama-3.2-3B, #Params=49.40M2025.0979.84——
MoRAModel=Llama-2-7B, Params (%)=0.82412026.0479.53——
HiRAModel=Llama-2-7B, Params (%)=0.82562026.0479.53——
LoRAModel=Llama-2-7B, Params (%)=0.82562026.0479.5——
PiSSAModel=Llama-3.2-3B, #Params=48.63M2025.0979.43——
rsLoRAModel=Llama-3.2-3B, #Params=48.63M2025.0978.92——
DoRAModel=Llama-2-7B, Params (%)=0.82562026.0476——
PiSSAModel=Llama-3.2-1B, #Params=22.54M2025.0975.54——
FFTModel=Llama-3.2-1B, #Params=1.24B2025.0975.37——
ABBAModel=Llama-3.2-1B, #Params=22.54M2025.0974.75——
GraLoRAModel=Llama-3.2-1B, #Params=22.54M2025.0973.95——
BoHAModel=Llama-3.2-1B, #Params=22.54M2025.0973.83——
rsLoRAModel=Llama-3.2-1B, #Params=22.54M2025.0973.47——
LoRAModel=Llama-3.2-1B, #Params=22.54M2025.0973.39——
HiRAModel=Llama-3.2-1B, #Params=22.54M2025.0973.39——
DoRAModel=Llama-3.2-1B, #Params=22.92M2025.0973.34——
RIDERSBackbone=GPT-3.52024.0272.129—
RIDERSModel=GPT-3.52024.0272.129—
RIDERSBackbone=Mistral-7B2024.0270.624.5—
Chain-of-ThoughtBackbone=GPT-3.52024.0270.630.3—
RIDERSModel=Mistral-7B2024.0270.624.5—
CoTModel=GPT-3.52024.0270.630.3—
Chain-of-ThoughtBackbone=Mistral-7B2024.0270.228.2—
CoTModel=Mistral-7B2024.0270.228.2—
SPS OnlyBackbone=Baichuan2-13B, Strategy=Serial-position Swap2024.0269.227.8—
RIDERSBackbone=Baichuan2-13B2024.026920.5—
RD OnlyBackbone=Baichuan2-13B, Strategy=Residual Decoding2024.0268.925—
Few-shot AnswerBackbone=Baichuan2-13B2024.0268.8——
ChatGPTModel=ChatGPT2026.0468.5——
Self-ConsistencyBackbone=Baichuan2-13B2024.0267.833.5—
Chain-of-ThoughtBackbone=Baichuan2-13B2024.0266.537.5—
Least-to-MostBackbone=Baichuan2-13B2024.0266.536.8—
Contrasive CoTBackbone=Baichuan2-13B2024.026537.3—
BaselineFormat=Baseline, Bit width (b)=16.00, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.0564.8——
Self-RefineBackbone=Baichuan2-13B2024.0262.243.1—
Tensor RMS + CFormat=Tensor RMS + C, Bit width (b)=3.00, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.0561.5——
Tensor RMS + SpFormat=Tensor RMS + Sp, Bit width (b)=3.05, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.0561.4——
Block AbsmaxFormat=Block Absmax, Bit width (b)=3.25, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.0559.5——
Channel AbsmaxFormat=Channel Absmax, Bit width (b)=3.00, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.0556.5——
MobileMoE-LActive Parameters=922M, Total Parameters=5.3B, Few-shot count=0-shot2026.0554.3——
Llama 3 405BParameters=405B2024.0753.7——
Qwen2.5-1.5B#Params=1.5B/1.5B, #Tokens=18T, Zero-shot=true2026.0253.5——
MobileMoE-LMem (GB)=2.75, INT4 QAT=true, Zero-shot=true2026.0552.9——
Llama 3 70BParameters=70B2024.0752.2——
Qwen3-1.7B#Params=1.7B/1.7B, #Tokens=36T, Zero-shot=true2026.0252.2——
GemmaSize=8B, Tokens=6T, Context=8K2024.0451.8——
Gemma 7BParameters=7B2024.0751.8——
Mixtral 8×22BParameters=8×22B2024.0751.6——
LLAMA2Size=13B, Tokens=2T, Context=4K2024.0450.3——
MobileMoE-MActive Parameters=528M, Total Parameters=2.8B, Few-shot count=0-shot2026.0550.2——
Tensor AbsmaxFormat=Tensor Absmax, Bit width (b)=3.00, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.0549.9——
MEGALODONSize=7B, Tokens=2T, Context=32K2024.0449.6——
Llama 3 8BParameters=8B2024.0749.5——
MobileMoE-MMem (GB)=1.48, INT4 QAT=true, Zero-shot=true2026.0549——
LLaMA-7BModel Category=LLM, Evaluation Mode=Zero-shot2023.0948.9——
DREAMLLM-7BModel Category=MLLM, Evaluation Mode=Zero-shot2023.0948.8——
MPTSize=7B, Tokens=1T, Context=4K2024.0448.5——
LLAMA2Size=7B, Tokens=2T, Context=4K2024.0448.3——
Mistral 7BParameters=7B2024.0748.2——
OLMoE-1B-7B#Params=1.3B/7B, #Tokens=5T, Zero-shot=true2026.0247.8——
Vicuna-7BModel Category=LLM, Evaluation Mode=Zero-shot2023.0947.5——
SPES-9B#Params=3.1B/9B, #Tokens=400B, Zero-shot=true, initialized from=pretrained dense model2026.0247.5——
MobileLLM-ProMem (GB)=0.55, INT4 QAT=true, Zero-shot=true2026.0547.4——
Qwen2.5-0.5B#Params=0.5B/0.5B, #Tokens=18T, Zero-shot=true2026.0247.1——
MistralSize=7B, Context=16K2024.0447——
Qwen3-0.6B#Params=0.6B/0.6B, #Tokens=36T, Zero-shot=true2026.0246.9——
MobileMoE-SActive Parameters=272M, Total Parameters=1.3B, Few-shot count=0-shot2026.0546.8——
SYNPROModel Size=1.1B, Alpha (organic token ratio)=10%2026.0546.78——
SmolLM2-1.7B#Params=1.7B/1.7B, #Tokens=11T, Zero-shot=true2026.0246.7——
Llama-3.1Size=8B, Type=AR2026.0146.7——
MoE++ 7B#Params=1.2B/7B, #Tokens=1T, Zero-shot=true2026.0245.7——
MobileMoE-SMem (GB)=0.68, INT4 QAT=true, Zero-shot=true2026.0545.5——
A3Size=8B, Type=A32026.0145.2——
Llama3.2-1B#Params=1.1B/1.1B, #Tokens=9T, Zero-shot=true2026.0245——
SPES-7B#Params=1.6B/7B, #Tokens=500B, Zero-shot=true2026.0244.8——
MobiLlama-1.3B#Params=1.3B/1.3B, #Tokens=1.3T, Zero-shot=true2026.0244.7——
Pythia-1.4B#Params=1.4B/1.4B, #Tokens=300B, Zero-shot=true2026.0244.6——
Pythia-2.8B#Params=2.8B/2.8B, #Tokens=300B, Zero-shot=true2026.0244.5——
OPT-2.7B#Params=2.7B/2.7B, #Tokens=180B, Zero-shot=true2026.0244.1——
OPT-1.3B#Params=1.3B/1.3B, #Tokens=180B, Zero-shot=true2026.0243.7——