List your company
ResearchFormerly WizWand

Physical Commonsense Reasoning on PIQA (val)

83AccuracyMistral-v0.1
66.98471.14275.379.458Jan 11, 2024Jun 2, 2024Oct 23, 2024Mar 15, 2025Aug 5, 2025Dec 26, 2025May 18, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Mistral-v0.1Zero-shot=true, Model Scale=7B2024.0183
ChatGPT + Self-consistent chain-of-thoughtZero-shot=true, Variant=gpt-3.5-turbo, Prompting Strategy=Self-consistent chain-of-thought2024.0181.7
KVQuant-4b-1%Backbone=Mistral-7b, Bits Per FPN=4.32-4.352024.0580.74
FP16Backbone=Mistral-7b, Bits Per FPN=162024.0580.58
KVQuant-4bBackbone=Mistral-7b, Bits Per FPN=4.00-4.022024.0580.58
CQ-2c8bBackbone=Mistral-7b, Bits Per FPN=4.002024.0580.52
DeBERTa-v3-L (CANDLE Distilled)Zero-shot=true, CSKB=CANDLE2024.0180.3
LLAMA2Zero-shot=true, Model Scale=13B2024.0180.2
LLM-Pruner+FinetuneBackbone=Llama-2-13b, Param Ratio=0.90, Fine-tuning=Alpaca2025.1279.7
KVQuant-2b-1%Backbone=Mistral-7b, Bits Per FPN=2.32-2.352024.0579.65
CQ-4c8bBackbone=Mistral-7b, Bits Per FPN=2.002024.0579.49
LLM-PrunerBackbone=Llama-2-13b, Param Ratio=0.90, Fine-tuning=None2025.1279.4
KVQuant-4b-1%Backbone=LLaMA-2-13b, Bits Per FPN=4.32-4.352024.0579.27
KVQuant-4bBackbone=LLaMA-13b, Bits Per FPN=4.00-4.022024.0579.22
ChatGPT + Chain-of-thoughtZero-shot=true, Variant=gpt-3.5-turbo, Prompting Strategy=Chain-of-thought2024.0179.2
LLM-Pruner+FinetuneBackbone=Llama-2-13b, Param Ratio=0.80, Fine-tuning=Alpaca2025.1279.2
FP16Backbone=LLaMA-13b, Bits Per FPN=162024.0579.16
FP16Backbone=LLaMA-2-13b, Bits Per FPN=162024.0579.16
KVQuant-4b-1%Backbone=LLaMA-13b, Bits Per FPN=4.32-4.352024.0579.16
CQ-2c8bBackbone=LLaMA-13b, Bits Per FPN=4.002024.0579.11
BaselineBackbone=Llama-2-13b, Param Ratio=1.00, Fine-tuning=None2025.1279.1
DeBERTa-v3-L (MR)Zero-shot=true, CSKB=ATM10X2024.0179
KVQuant-4bBackbone=LLaMA-2-13b, Bits Per FPN=4.00-4.022024.0578.94
LLAMA2Zero-shot=true, Model Scale=7B2024.0178.8
FP16Backbone=LLaMA-7b, Bits Per FPN=162024.0578.67
KVQuant-4bBackbone=LLaMA-7b, Bits Per FPN=4.00-4.022024.0578.62
CQ-2c8bBackbone=LLaMA-2-13b, Bits Per FPN=4.002024.0578.62
CQ-2c8bBackbone=LLaMA-7b, Bits Per FPN=4.002024.0578.61
CAR-DeBERTa-v3-LZero-shot=true, CSKB=ATOMIC2024.0178.6
CAR-DeBERTa-v3-LZero-shot=true, CSKB=AbsATM2024.0178.6
LLRCBackbone=Llama-2-13b, Param Ratio=0.90, Fine-tuning=None2025.1278.6
KVQuant-2b-1%Backbone=LLaMA-13b, Bits Per FPN=2.32-2.352024.0578.51
KVQuant-2b-1%Backbone=LLaMA-2-13b, Bits Per FPN=2.32-2.352024.0578.51
KVQuant-4b-1%Backbone=LLaMA-7b, Bits Per FPN=4.32-4.352024.0578.4
LLM-PrunerBackbone=Llama-2-13b, Param Ratio=0.80, Fine-tuning=None2025.1278.3
CQ-4c8bBackbone=LLaMA-13b, Bits Per FPN=2.002024.0578.29
BaselineBackbone=Llama-2-7b, Param Ratio=1.00, Fine-tuning=None2025.1278.1
LLRCBackbone=Llama-2-7b, Param Ratio=0.90, Fine-tuning=None2025.1278.1
LLM-Pruner+FinetuneBackbone=Llama-2-7b, Param Ratio=0.90, Fine-tuning=Alpaca2025.1278.1
FP16Backbone=LLaMA-2-7b, Bits Per FPN=162024.0578.07
KVQuant-4b-1%Backbone=LLaMA-2-7b, Bits Per FPN=4.32-4.352024.0578.07
DeBERTa-v3-L (MR)Zero-shot=true, CSKB=ATOMIC2024.0178
CQ-2c8bBackbone=LLaMA-2-7b, Bits Per FPN=4.002024.0577.91
LLRCBackbone=Llama-2-13b, Param Ratio=0.80, Fine-tuning=None2025.1277.9
KVQuant-4bBackbone=LLaMA-2-7b, Bits Per FPN=4.00-4.022024.0577.86
LLM-Pruner+FinetuneBackbone=Llama-2-7b, Param Ratio=0.80, Fine-tuning=Alpaca2025.1277.8
KVQuant-2b-1%Backbone=LLaMA-7b, Bits Per FPN=2.32-2.352024.0577.69
VERA-T5-xxl (CANDLE Distilled)Zero-shot=true, CSKB=CANDLE2024.0177.6
SVD-LLM+FinetuneBackbone=Llama-2-13b, Param Ratio=0.90, Fine-tuning=Alpaca2025.1277.6
CQ-4c8bBackbone=LLaMA-2-13b, Bits Per FPN=2.002024.0577.42
CQ-8c10bBackbone=Mistral-7b, Bits Per FPN=1.252024.0577.31
LLM-PrunerBackbone=Llama-2-7b, Param Ratio=0.90, Fine-tuning=None2025.1277.3
VERA-T5-xxlZero-shot=true, CSKB=AbsATM2024.0177.2
KVQuant-2b-1%Backbone=LLaMA-2-7b, Bits Per FPN=2.32-2.352024.0576.6
VERA-T5-xxlZero-shot=true, CSKB=ATOMIC2024.0176.4
SVD-LLM+FinetuneBackbone=Llama-2-13b, Param Ratio=0.80, Fine-tuning=Alpaca2025.1276.3
LLM-PrunerBackbone=Llama-2-7b, Param Ratio=0.80, Fine-tuning=None2025.1276.2
CQ-4c8bBackbone=LLaMA-2-7b, Bits Per FPN=2.002024.0576.12
CQ-4c8bBackbone=LLaMA-7b, Bits Per FPN=2.002024.0576.11
CQ-8c10bBackbone=LLaMA-13b, Bits Per FPN=1.252024.0575.9
KVQuant-2bBackbone=Mistral-7b, Bits Per FPN=2.00-2.022024.0575.46
KVQuant-1b-1%Backbone=LLaMA-13b, Bits Per FPN=1.32-1.352024.0575.46
SVD-LLM+FinetuneBackbone=Llama-2-7b, Param Ratio=0.90, Fine-tuning=Alpaca2025.1275.3
CQ-8c8bBackbone=Mistral-7b, Bits Per FPN=1.002024.0575.24
ChatGPTZero-shot=true, Variant=gpt-3.5-turbo2024.0175.1
VERA-T5-xxlZero-shot=true, CSKB=ATM10X2024.0175.1
LLRCBackbone=Llama-2-7b, Param Ratio=0.80, Fine-tuning=None2025.1275.1
KVQuant-2bBackbone=LLaMA-13b, Bits Per FPN=2.00-2.022024.0574.81
CQ-8c10bBackbone=LLaMA-2-13b, Bits Per FPN=1.252024.0574.37
CQ-8c8bBackbone=LLaMA-13b, Bits Per FPN=1.002024.0573.99
SVD-LLM (w)Backbone=Llama-2-13b, Param Ratio=0.90, Fine-tuning=None2025.1273.9
KVQuant-1b-1%Backbone=Mistral-7b, Bits Per FPN=1.32-1.352024.0573.83
CQ-8c10bBackbone=LLaMA-7b, Bits Per FPN=1.252024.0573.45
CQ-8c10bBackbone=LLaMA-2-7b, Bits Per FPN=1.252024.0573.07
CQ-8c8bBackbone=LLaMA-2-13b, Bits Per FPN=1.002024.0573.01
GPT-4Zero-shot=true, Variant=gpt-42024.0173
KVQuant-2bBackbone=LLaMA-7b, Bits Per FPN=2.00-2.022024.0572.47
SVD-LLM+FinetuneBackbone=Llama-2-7b, Param Ratio=0.80, Fine-tuning=Alpaca2025.1272.4
DeepSeekMoE# Shot=0-shot, # Total Params=2.0B, # Activated Params=0.3B, FLOPs per 2K Tokens=4.3T, # Training Tokens=100B2024.0172.3
KVQuant-1b-1%Backbone=LLaMA-7b, Bits Per FPN=1.32-1.352024.0571.38
CQ-8c8bBackbone=LLaMA-2-7b, Bits Per FPN=1.002024.0571.22
CQ-8c8bBackbone=LLaMA-7b, Bits Per FPN=1.002024.0571.16
STL-AdapterZero-shot=true, CSKB=ATOMIC2024.0171.1
KVQuant-1b-1%Backbone=LLaMA-2-13b, Bits Per FPN=1.32-1.352024.0570.89
GShard# Shot=0-shot, # Total Params=2.0B, # Activated Params=0.3B, FLOPs per 2K Tokens=4.3T, # Training Tokens=100B2024.0170.6
Switch# Shot=0-shot, # Total Params=2.0B, # Activated Params=0.2B, FLOPs per 2K Tokens=2.9T, # Training Tokens=100B2024.0170.5
AMOOptimizer=AMO, Model=Llama3.1-1.4B, Few-shot=02026.0570.5
Mistral (Full-Attention)Model Scale=1.4B2024.0770.4
Mamba (SSM)Model Scale=1.4B2024.0770.3
Self-talkZero-shot=true2024.0170.2
BMoJo (Fading)Model Scale=1.4B2024.0770
KVQuant-1b-1%Backbone=LLaMA-2-7b, Bits Per FPN=1.32-1.352024.0569.91
BMoJo (Fading + Eidetic)Model Scale=1.4B2024.0769.9
SVD-LLM (w)Backbone=Llama-2-13b, Param Ratio=0.80, Fine-tuning=None2025.1269.8
SVD-LLM (w)Backbone=Llama-2-7b, Param Ratio=0.90, Fine-tuning=None2025.1269.4
SmolLM Ceiling (O(L^2))Parameters=135M, Training Tokens=∼600 Billion, Evaluation Mode=Zero-shot2026.0468.55
Hash Layer# Shot=0-shot, # Total Params=2.0B, # Activated Params=0.2B, FLOPs per 2K Tokens=2.9T, # Training Tokens=100B2024.0168.4
GPT 3.5Zero-shot=true, Variant=text-davinci-0032024.0167.8
AMOOptimizer=AMO, Model=Llama3.1-760M, Few-shot=02026.0567.7
ROBERTa-LZero-shot=true2024.0167.6