Question Answering on MS MARCO MQA (test val)
69.4F1 ScoreIN-CONTEXT
Evaluation results
| Method | Links | |
|---|---|---|
| IN-CONTEXTAMORTIZABLE TIME (S)=0, GENERATION TIME (S)=14.2, context_usage=full context2026.02 | 69.4 | |
| SHINEAMORTIZABLE TIME (S)=0.3, GENERATION TIME (S)=11, hypernetwork=single forward pass2026.02 | 55.6 | |
| SFTAMORTIZABLE TIME (S)=29.3, GENERATION TIME (S)=11, fine-tuning=rank=8 LoRA, epochs=102026.02 | 33 | |
| NAIVEAMORTIZABLE TIME (S)=0, GENERATION TIME (S)=11, context_usage=none2026.02 | 23.2 |