Reading Comprehension on MultiRC (Accuracy)
72.9MultiRC AccuracyGPT-3
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-3Shots=12023.03 | 72.9 | |
| MUPPET2022.12 | 64.15 | |
| ColD-Fusion2022.12 | 63.01 | |
| Multitask2022.12 | 62.34 | |
| BloombergGPTShots=12023.03 | 62.29 | |
| Finetune2022.12 | 60.56 | |
| DenseBase Model=DeepSeek-7B, Sparsity=Dense2025.06 | 57.22 | |
| SparsegptBase Model=LLaMA-2-7B, Sparsity=2:42025.06 | 57.2 | |
| SparsegptBase Model=DeepSeek-7B, Sparsity=2:42025.06 | 57.2 | |
| Pruner-ZBase Model=DeepSeek-7B, Sparsity=2:42025.06 | 57.2 | |
| MaskProBase Model=DeepSeek-7B, Sparsity=2:42025.06 | 57.2 | |
| DenseBase Model=LLaMA-2-7B, Sparsity=Dense2025.06 | 56.97 | |
| MaskProBase Model=LLaMA-2-7B, Sparsity=2:42025.06 | 56.93 | |
| RegMixBackbone=TinyLlama-120M, Training Budget=50B tokens, FLOPs overhead=3.072 × 10^182026.04 | 56.7 | |
| UniformBackbone=TinyLlama-120M, Training Budget=50B tokens, FLOPs overhead=02026.04 | 56.68 | |
| ADAPT-BM25Backbone=TinyLlama-120M, Training Budget=50B tokens, FLOPs overhead=≪ 1.0 × 10^142026.04 | 56.53 | |
| Pruner-ZBase Model=LLaMA-2-7B, Sparsity=2:42025.06 | 56.37 | |
| ADAPTBackbone=TinyLlama-120M, Training Budget=50B tokens, FLOPs overhead=≪ 1.1 × 10^152026.04 | 55.67 | |
| Human2026.06 | 54.15 | |
| DoReMi2026.06 | 53.31 | |
| RegMix2026.06 | 52.86 | |
| RegMix-DProxy models=512, Mode=Offline2026.06 | 52.13 | |
| RegMix-DProxy models=512, Mode=Online2026.06 | 51.71 | |
| RegMix-DProxy models=128, Mode=Online2026.06 | 51.57 | |
| LinUpperBackbone=TinyLlama-120M, Training Budget=50B tokens, FLOPs overhead=02026.04 | 50.74 | |
| RegMix-DProxy models=128, Mode=Offline2026.06 | 50.25 | |
| DoReMiBackbone=TinyLlama-120M, Training Budget=50B tokens, FLOPs overhead=4.92 × 10^192026.04 | 48.99 | |
| SSAArchitecture=Nemotron-style decoder, Parameters=114M, Training steps=22K, Training dataset=FineWeb, Zero-shot=true2025.08 | 43.5 | |
| SoftmaxArchitecture=Nemotron-style decoder, Parameters=114M, Training steps=22K, Training dataset=FineWeb, Zero-shot=true2025.08 | 42.8 | |
| BLOOM176BShots=12023.03 | 26.65 | |
| GPT-NeoXShots=12023.03 | 22.86 | |
| OPT66BShots=12023.03 | 18.8 |