Multiple Choice Question Answering on MMLU (5-shot)
73.4AccuracyEdgeJury
Evaluation Results
| Method | Links | |
|---|---|---|
| EdgeJuryvariant=Full, Shots=5-shot2025.12 | 73.4 | |
| RAG-S1Retrieval-Augmentation=true, Shots=5-shot2025.12 | 70.5 | |
| FP16Model=LLaMA2-70B, Bit-width=16-bit, Quantization Method=None, Group Size=None2023.11 | 69.58 | |
| SC5k=5, Shots=5-shot2025.12 | 69.3 | |
| NF3-symModel=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.11 | 66.92 | |
| SC3k=3, Shots=5-shot2025.12 | 66.8 | |
| AFPQ (NF3-asym)Model=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.11 | 66.78 | |
| AFPQ (NF3-asym)Model=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.11 | 66.43 | |
| NF3-symModel=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.11 | 66.4 | |
| MVstrategy=Majority Vote, Shots=5-shot2025.12 | 66.4 | |
| AFPQ (NF3-asym)Model=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.11 | 66.23 | |
| INT3Model=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.11 | 66.16 | |
| NF3-symModel=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.11 | 65.9 | |
| INT3Model=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.11 | 65.05 | |
| INT3Model=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.11 | 64.77 | |
| S1k=1, Shots=5-shot2025.12 | 64.2 | |
| AFPQ (NF3-asym)Model=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.11 | 56.07 | |
| FP16Model=LLaMA2-13B, Bit-width=16-bit, Quantization Method=None, Group Size=None2023.11 | 55.38 | |
| AFPQ (NF3-asym)Model=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.11 | 53.3 | |
| AFPQ (NF3-asym)Model=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.11 | 52.6 | |
| AFPQ (NF3-asym)Model=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.11 | 52.31 | |
| NF3-symModel=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.11 | 51.72 | |
| INT3Model=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.11 | 51.23 | |
| NF3-symModel=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.11 | 50.97 | |
| INT3Model=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.11 | 50.77 | |
| NF3-symModel=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.11 | 49.84 | |
| INT3Model=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.11 | 48.91 | |
| FullBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.03 | 46.64 | |
| Filter-then-WeightBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.03 | 46.63 | |
| FullBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.03 | 46.59 | |
| FP16Model=LLaMA2-7B, Bit-width=16-bit, Quantization Method=None, Group Size=None2023.11 | 46.58 | |
| LESSBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.03 | 46.56 | |
| TracIncBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.03 | 46.45 | |
| Filter-then-WeightBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.03 | 46.23 | |
| TracIncBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.03 | 46.16 | |
| GREATSBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.03 | 46.06 | |
| GREATSBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.03 | 46.06 | |
| LESSBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.03 | 45.98 | |
| OriginBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.03 | 45.86 | |
| OriginBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.03 | 45.86 | |
| GRAD-MATCHBackbone=Qwen3-0.6B, Selection Protocol=Best-of-run2026.03 | 45.72 | |
| GRAD-MATCHBackbone=Qwen3-0.6B, Selection Protocol=Fixed-budget2026.03 | 44.98 | |
| AFPQ (NF3-asym)Model=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.11 | 42.74 | |
| AFPQ (NF3-asym)Model=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.11 | 41.61 | |
| AFPQ (NF3-asym)Model=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.11 | 41.11 | |
| NF3-symModel=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.11 | 40.63 | |
| INT3Model=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.11 | 40.06 | |
| NF3-symModel=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.11 | 38.96 | |
| NF3-symModel=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.11 | 38.61 | |
| AFPQ (NF3-asym)Model=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.11 | 38.58 | |
| INT3Model=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g1282023.11 | 38.5 | |
| NF3-symModel=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g642023.11 | 38.47 | |
| INT3Model=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.11 | 37.46 | |
| NF3-symModel=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g2562023.11 | 36.85 | |
| INT3Model=LLaMA2-70B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.11 | 34.39 | |
| Filter-then-WeightBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.03 | 30.93 | |
| Filter-then-WeightBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.03 | 30.93 | |
| AFPQ (NF3-asym)Model=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.11 | 30.31 | |
| GRAD-MATCHBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.03 | 30.14 | |
| GREATSBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.03 | 30.07 | |
| LESSBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.03 | 29.74 | |
| LESSBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.03 | 29.67 | |
| GREATSBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.03 | 29.59 | |
| TracIncBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.03 | 29.24 | |
| FullBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.03 | 29.09 | |
| TracIncBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.03 | 29.04 | |
| GRAD-MATCHBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.03 | 28.77 | |
| FullBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.03 | 28.67 | |
| OriginBackbone=Llama-3.2-1B, Selection Protocol=Best-of-run2026.03 | 28.13 | |
| OriginBackbone=Llama-3.2-1B, Selection Protocol=Fixed-budget2026.03 | 28.13 | |
| INT3Model=LLaMA2-13B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.11 | 27.79 | |
| NF3-symModel=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.11 | 26.2 | |
| INT3Model=LLaMA2-7B, Bit-width=3-bit, Quantization Method=RTN, Group Size=g-12023.11 | 25.22 |