List your company
ResearchFormerly WizWand

Multiple Choice Question Answering on MMLU (5-shot)

73.4AccuracyEdgeJury
23.292836.301449.3162.3186Nov 3, 2023Mar 24, 2024Aug 14, 2024Jan 4, 2025May 26, 2025Oct 16, 2025Mar 8, 2026
Updated 7mo ago

Evaluation results

MethodLinks
EdgeJuryvariant=Full, Shots=5-shot2025.1273.4
RAG-S1Retrieval-Augmentation=true, Shots=5-shot2025.1270.5
FP16Model=LLaMA2-70B, Bit-width=16-bit, Quantization Method=None, Group Size=None2023.1169.58
SC5k=5, Shots=5-shot2025.1269.3
NF3-symModel=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.1166.92
SC3k=3, Shots=5-shot2025.1266.8
AFPQ (NF3-asym)Model=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.1166.78
AFPQ (NF3-asym)Model=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.1166.43
NF3-symModel=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.1166.4
MVstrategy=Majority Vote, Shots=5-shot2025.1266.4
AFPQ (NF3-asym)Model=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.1166.23
INT3Model=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.1166.16
NF3-symModel=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.1165.9
INT3Model=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.1165.05
INT3Model=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.1164.77
S1k=1, Shots=5-shot2025.1264.2
AFPQ (NF3-asym)Model=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.1156.07
FP16Model=LLaMA2-13B, Bit-width=16-bit, Quantization Method=None, Group Size=None2023.1155.38
AFPQ (NF3-asym)Model=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.1153.3
AFPQ (NF3-asym)Model=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.1152.6
AFPQ (NF3-asym)Model=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.1152.31
NF3-symModel=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.1151.72
INT3Model=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.1151.23
NF3-symModel=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.1150.97
INT3Model=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.1150.77
NF3-symModel=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.1149.84
INT3Model=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.1148.91
FullBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.0346.64
Filter-then-WeightBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.0346.63
FullBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.0346.59
FP16Model=LLaMA2-7B, Bit-width=16-bit, Quantization Method=None, Group Size=None2023.1146.58
LESSBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.0346.56
TracIncBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.0346.45
Filter-then-WeightBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.0346.23
TracIncBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.0346.16
GREATSBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.0346.06
GREATSBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.0346.06
LESSBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.0345.98
OriginBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.0345.86
OriginBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.0345.86
GRAD-MATCHBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.0345.72
GRAD-MATCHBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.0344.98
AFPQ (NF3-asym)Model=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.1142.74
AFPQ (NF3-asym)Model=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.1141.61
AFPQ (NF3-asym)Model=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.1141.11
NF3-symModel=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.1140.63
INT3Model=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.1140.06
NF3-symModel=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.1138.96
NF3-symModel=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.1138.61
AFPQ (NF3-asym)Model=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.1138.58
INT3Model=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.1138.5
NF3-symModel=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.1138.47
INT3Model=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.1137.46
NF3-symModel=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.1136.85
INT3Model=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.1134.39
Filter-then-WeightBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.0330.93
Filter-then-WeightBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.0330.93
AFPQ (NF3-asym)Model=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.1130.31
GRAD-MATCHBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.0330.14
GREATSBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.0330.07
LESSBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.0329.74
LESSBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.0329.67
GREATSBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.0329.59
TracIncBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.0329.24
FullBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.0329.09
TracIncBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.0329.04
GRAD-MATCHBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.0328.77
FullBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.0328.67
OriginBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.0328.13
OriginBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.0328.13
INT3Model=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.1127.79
NF3-symModel=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.1126.2
INT3Model=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.1125.22