Reading Comprehension on BoolQ (Accuracy, Retention)
74.1AccuracyDense Baseline
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Dense BaselineBackbone=Llama-2-7B2026.03 | 74.1 | 100 | |
| Standard Top-K (Oracle)Backbone=Llama-2-7B2026.03 | 73.9 | 99.7 | |
| AFBS-BOBackbone=Llama-2-7B2026.03 | 73.8 | 99.6 | |
| H2OBackbone=Llama-2-7B2026.03 | 73.5 | 99.2 | |
| Routing TransformerBackbone=Llama-2-7B2026.03 | 72.8 | 98.2 | |
| Window AttentionBackbone=Llama-2-7B2026.03 | 69.8 | 94.2 |