Reading Comprehension on RACE MRQA out-of-domain
43.62EMPrompt tuning
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Prompt tuningBackbone=Llama-3.2-3B-Base, Context=with context2026.02 | 43.62 | 56.35 | |
| Prompt tuningBackbone=Llama-3.2-1B-Base, Context=with context2026.02 | 33.68 | 46.3 | |
| ComprExITBackbone=Llama-3.2-3B-Base2026.02 | 31.6 | 47.69 | |
| BeaconBackbone=Llama-3.2-3B-Base2026.02 | 30.71 | 43.95 | |
| ComprExITBackbone=Llama-3.2-1B-Base2026.02 | 23 | 37.95 | |
| Zero-shotBackbone=Llama-3.2-3B-Base, Context=with context2026.02 | 22.7 | 38.16 | |
| ICAEBackbone=Llama-3.2-3B-Base2026.02 | 21.51 | 33.85 | |
| 500xBackbone=Llama-3.2-3B-Base2026.02 | 19.14 | 30.58 | |
| BeaconBackbone=Llama-3.2-1B-Base2026.02 | 15.88 | 25.35 | |
| Zero-shotBackbone=Llama-3.2-1B-Base, Context=with context2026.02 | 10.53 | 22.71 | |
| ICAEBackbone=Llama-3.2-1B-Base2026.02 | 9.5 | 16.95 | |
| 500xBackbone=Llama-3.2-1B-Base2026.02 | 1.93 | 5.7 | |
| Zero-shotBackbone=Llama-3.2-3B-Base, Context=without context2026.02 | 1.78 | 5.65 | |
| Zero-shotBackbone=Llama-3.2-1B-Base, Context=without context2026.02 | 0.45 | 2.88 |