Multiple Choice Question Answering on OpenBookQA
44.4AccuracyOriginal
Evaluation results
| Method | Links | |
|---|---|---|
| OriginalBackbone=Mistral-7B, Pruning Ratio=0%2026.05 | 44.4 | |
| AccBackbone=Mistral-7B, Pruning Ratio=25%, Relevance Estimation Source=Task-specific2026.05 | 42.4 | |
| Out. Cosine-SimBackbone=Mistral-7B, Pruning Ratio=25%, Relevance Estimation Source=Task-specific2026.05 | 37.8 | |
| Out. Norm-SimBackbone=Mistral-7B, Pruning Ratio=25%, Relevance Estimation Source=Task-specific2026.05 | 37.8 | |
| Cosine SimilarityBackbone=Mistral-7B, Pruning Ratio=25%, Relevance Estimation Source=Task-specific2026.05 | 37.4 | |
| ExoFormerAnchor Source=External Anchor, Mixing Granularity=Elementwise, Normalization Strategy=Full2026.01 | 36.4 | |
| PerplexityBackbone=Mistral-7B, Pruning Ratio=25%, Relevance Estimation Source=Task-specific2026.05 | 36.4 | |
| NuResFormerAnchor Source=Internal Anchor, Mixing Granularity=Elementwise, Normalization Strategy=No Norm, Mixing Path Components={V,G}2026.01 | 35 | |
| NuResFormerAnchor Source=Internal Anchor, Mixing Granularity=Scalar, Normalization Strategy=No Norm, Mixing Path Components={V,G}2026.01 | 34.8 | |
| ResFormer (Value Residual)Anchor Source=Baselines2026.01 | 34.4 | |
| ExoFormerAnchor Source=External Anchor, Mixing Granularity=Elementwise, Normalization Strategy=No Norm2026.01 | 34.4 | |
| Dynamic ExoFormerAnchor Source=External Anchor, Mixing Granularity=Elementwise, Normalization Strategy=Full2026.01 | 34.4 | |
| Out. Divergence-SimBackbone=Mistral-7B, Pruning Ratio=25%, Relevance Estimation Source=Task-specific2026.05 | 34.4 | |
| NuResFormerAnchor Source=Internal Anchor, Mixing Granularity=Headwise, Normalization Strategy=Only Q/K Norms2026.01 | 34.2 | |
| NuResFormerAnchor Source=Internal Anchor, Mixing Granularity=Scalar, Normalization Strategy=Only Q/K Norms2026.01 | 34.2 | |
| Gated AttentionAnchor Source=Baselines2026.01 | 34 | |
| NuResFormerAnchor Source=Internal Anchor, Mixing Granularity=Headwise, Normalization Strategy=No Norm, Mixing Path Components={V,G}2026.01 | 34 | |
| Base TransformerAnchor Source=Baselines2026.01 | 33.6 | |
| NuResFormerAnchor Source=Internal Anchor, Mixing Granularity=Elementwise, Normalization Strategy=Full2026.01 | 33.6 | |
| ExoFormerAnchor Source=External Anchor, Mixing Granularity=Scalar, Normalization Strategy=Full2026.01 | 33.6 | |
| NuResFormerAnchor Source=Internal Anchor, Mixing Granularity=Headwise, Normalization Strategy=Full2026.01 | 33.4 | |
| NuResFormerAnchor Source=Internal Anchor, Mixing Granularity=Scalar, Normalization Strategy=Full2026.01 | 33.2 | |
| NuResFormerAnchor Source=Internal Anchor, Mixing Granularity=Elementwise, Normalization Strategy=Only Q/K Norms2026.01 | 33 | |
| Naïve CombinationAnchor Source=Baselines2026.01 | 32.6 | |
| ExoFormerAnchor Source=External Anchor, Mixing Granularity=Headwise, Normalization Strategy=Full2026.01 | 32.4 | |
| R2LMShots=0-sh, Candidate Type=long-target2026.06 | 27.4 | |
| Causal dLLMShots=0-sh, Candidate Type=long-target2026.06 | 27 | |
| R2LM-PI (plug-in)Shots=0-sh, Candidate Type=long-target2026.06 | 25.6 | |
| TaylorBackbone=Mistral-7B, Pruning Ratio=25%, Relevance Estimation Source=Task-specific2026.05 | 24.4 | |
| Bidirectional dLLMShots=0-sh, Candidate Type=long-target2026.06 | 16 |