Multiple-choice Question Answering on BoolQ
0.887MC AccuracyOriginal
Evaluation Results
| Method | Links | |
|---|---|---|
| OriginalModel=Qwen3-30B-A3B, Reduction ratio=0%2026.05 | 0.887 | |
| FrequencyModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=Pruning2026.05 | 0.885 | |
| ConMoEModel=Qwen3-30B-A3B, Reduction ratio=25%2026.05 | 0.885 | |
| M-SMoEModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=Merging2026.05 | 0.884 | |
| REAPModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=Pruning2026.05 | 0.881 | |
| FrequencyModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=Pruning2026.05 | 0.874 | |
| REAPModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=Pruning2026.05 | 0.861 | |
| HC-SMoEModel=Qwen3-30B-A3B, Reduction ratio=25%, Compression type=Merging2026.05 | 0.859 | |
| ConMoEModel=Qwen3-30B-A3B, Reduction ratio=50%2026.05 | 0.851 | |
| Spherical SteeringZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.02 | 0.8294 | |
| CAAZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.02 | 0.8242 | |
| M-SMoEModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=Merging2026.05 | 0.818 | |
| ITIZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.02 | 0.8153 | |
| SADI-HEADZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.02 | 0.802 | |
| BaselineZero-shot=true, Backbone=LLaMA-3.1-8B-Instruct2026.02 | 0.8012 | |
| HC-SMoEModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=Merging2026.05 | 0.755 | |
| ConMoEModel=deepseek-moe-16b-base, Reduction ratio=25%2026.05 | 0.745 | |
| OriginalModel=deepseek-moe-16b-base, Reduction ratio=0%2026.05 | 0.74 | |
| M-SMoEModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=Merging2026.05 | 0.738 | |
| HC-SMoEModel=Qwen3-30B-A3B, Reduction ratio=50%, Compression type=Merging2026.05 | 0.734 | |
| AR (autoregressive)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.05 | 0.719 | |
| OriginalModel=OLMoE-1B-7B-0125, Reduction ratio=0%2026.05 | 0.704 | |
| FrequencyModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=Pruning2026.05 | 0.699 | |
| ConMoEModel=deepseek-moe-16b-base, Reduction ratio=50%2026.05 | 0.697 | |
| REAPModel=deepseek-moe-16b-base, Reduction ratio=25%, Compression type=Pruning2026.05 | 0.692 | |
| Bidirectional dLLMShots=0-sh, Candidate Type=short-target2026.06 | 0.684 | |
| HC-SMoEModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=Merging2026.05 | 0.678 | |
| R2LM-PI (plug-in)Shots=0-sh, Candidate Type=short-target2026.06 | 0.673 | |
| R2LMShots=0-sh, Candidate Type=short-target2026.06 | 0.67 | |
| HC-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=Merging2026.05 | 0.659 | |
| Causal dLLMShots=0-sh, Candidate Type=short-target2026.06 | 0.653 | |
| FrequencyModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=Pruning2026.05 | 0.643 | |
| AR (autoregressive, ours)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.05 | 0.636 | |
| FrequencyModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=Pruning2026.05 | 0.635 | |
| REAPModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=Pruning2026.05 | 0.631 | |
| M-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=Merging2026.05 | 0.631 | |
| MDLM (masked diffusion)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.05 | 0.628 | |
| M-SMoEModel=deepseek-moe-16b-base, Reduction ratio=50%, Compression type=Merging2026.05 | 0.619 | |
| ConMoEModel=OLMoE-1B-7B-0125, Reduction ratio=25%2026.05 | 0.618 | |
| SMDM-1BZero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1B2026.05 | 0.615 | |
| Duo (uniform diffusion)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.05 | 0.596 | |
| REAPModel=OLMoE-1B-7B-0125, Reduction ratio=25%, Compression type=Pruning2026.05 | 0.589 | |
| ConMoEModel=OLMoE-1B-7B-0125, Reduction ratio=50%2026.05 | 0.579 | |
| HC-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=Merging2026.05 | 0.578 | |
| FrequencyModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=Pruning2026.05 | 0.573 | |
| Eso-LM (interpolating)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.05 | 0.534 | |
| ChanceZero-shot=true, Likelihood-based evaluation=true2026.05 | 0.504 | |
| REAPModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=Pruning2026.05 | 0.499 | |
| M-SMoEModel=OLMoE-1B-7B-0125, Reduction ratio=50%, Compression type=Merging2026.05 | 0.429 | |
| CFM (ours)Zero-shot=true, Likelihood-based evaluation=true, Number of Parameters=1.7B2026.05 | 0.425 |